{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loss Functions\n",
    "\n",
    "This python script illustrates the different loss functions for regression and classification.\n",
    "\n",
    "We start by loading the ncessary libraries and resetting the computational graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a Graph Session"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numerical Predictions\n",
    "\n",
    "---------------------------------\n",
    "\n",
    "To start with our investigation of loss functions, we begin by looking at numerical loss functions.  To do so, we must create a sequence of predictions around a target.  For this exercise, we consider the target to be zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Various Predicted X-values\n",
    "x_vals = tf.linspace(-1., 1., 500)\n",
    "\n",
    "# Create our target of zero\n",
    "target = tf.constant(0.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L2 Loss  -------->  Regression\n",
    "\n",
    "The L2 loss is one of the most common regression loss functions.  Here we show how to create it in TensorFlow and we evaluate it for plotting later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# L2 loss\n",
    "# L = (pred - actual)^2\n",
    "l2_y_vals = tf.square(target - x_vals)\n",
    "l2_y_out = sess.run(l2_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L1 Loss -------->  Regression\n",
    "\n",
    "An alternative loss function to consider is the L1 loss. This is very similar to L2 except that we take the `absolute value` of the difference instead of squaring it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# L1 loss\n",
    "# L = abs(pred - actual)\n",
    "l1_y_vals = tf.abs(target - x_vals)\n",
    "l1_y_out = sess.run(l1_y_vals)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pseudo-Huber Loss -------->  Regression\n",
    "$$ {\\displaystyle L_{\\delta }(a)=\\delta ^{2}({\\sqrt {1+(a/\\delta )^{2}}}-1).} $$\n",
    "The psuedo-huber loss function is a smooth approximation to the L1 loss as the (predicted - target) values get larger.  When the predicted values are close to the target, the pseudo-huber loss behaves similar to the L2 loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# L = delta^2 * (sqrt(1 + ((pred - actual)/delta)^2) - 1)\n",
    "\n",
    "# Pseudo-Huber with delta = 0.25\n",
    "delta1 = tf.constant(0.25)\n",
    "phuber1_y_vals = tf.multiply(tf.square(delta1), tf.sqrt(1. + tf.square((target - x_vals)/delta1)) - 1.)\n",
    "phuber1_y_out = sess.run(phuber1_y_vals)\n",
    "\n",
    "# Pseudo-Huber with delta = 5\n",
    "delta2 = tf.constant(5.)\n",
    "phuber2_y_vals = tf.multiply(tf.square(delta2), tf.sqrt(1. + tf.square((target - x_vals)/delta2)) - 1.)\n",
    "phuber2_y_out = sess.run(phuber2_y_vals)\n",
    "\n",
    "detal3 = tf.constant(3.)\n",
    "phuber3_y_val =tf.multiply(tf.square(detal3),tf.subtract(tf.sqrt(tf.add(1.,tf.square((target-x_vals)/detal3))),1.))\n",
    "phuber3_y_out=sess.run(phuber3_y_val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Regression Losses\n",
    "\n",
    "Here we use Matplotlib to plot the L1, L2, and Pseudo-Huber Losses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VNXWh9+dAgldRIp0MYAIJEgNIOV+qCAISlGQJlKk\nKXoBES+iXmxgoZcEVKQGRKVJ8UoRkKCA1IBARJQOUpSQAElmf3/sTGYmmZTJlHOS7Pd58iQzOefs\nNZnJXmevvdZvCSklGo1Go8l/+BltgEaj0WiMQTsAjUajyadoB6DRaDT5FO0ANBqNJp+iHYBGo9Hk\nU7QD0Gg0mnyKRxyAEKKtEOKYECJWCPFaJsc1FEIkCSG6emJcjUaj0eQctx2AEMIfmAm0A2oBPYQQ\ntTI4biLwnbtjajQajcZ9PLECaATESilPSinvAFFAJyfHvQh8BVzywJgajUajcZMAD1yjPHDa7vEZ\noLH9AUKI8sBTQGugYWYXE0IMAgYBBAcH169YsWKOjLJYLPj5Zd+/Xb5ckGvXCqQ+LlYinrKlk3M0\ntiftSn++4LffilC8eCKlS98y3K7g8+fxv3mTm/fdh3TjdXnaLm+T3+wSycn437pFUuHCOTrfk3Zd\nuBBEXFwA1arFIYR71/LW3+v8JT9uXLf9re666w733HPbJ3YdP378LynlPdk6WErp1hfQFZhn97g3\nMCPNMV8CTVJ+ng90zc6169evL3PKli1bXDr+vfekFEJKkBIs0s/PIiMicjy8x+xyRufOUpYrJ2Vy\nsvv2WMmxXT/9JOX770t544bnjLHDE38vb6Dtcg1P2XX7tpTFi0vZt69HLueVv1dEhJR+fta5RM0r\n773nO7uAPTKb87cnXN9ZwP42vULKc/Y0AKKEEKdSHMYsIcSTHhjbY7RqBf7+1kcCi0UwdChERxto\nVAZ07Qrnz8POnUZbAjRqBK+9BkWKGG2JxhtYLNCjB2zaZLQlAHz/Pfz9N3TpYrQlzolceZDBQ5Kx\nWGwaawEBan4xI55wALuBECFEVSFEAaA7sNr+ACllVSllFSllFWAFMFRKudIDY3uM8HCYOROHJWVy\nsmTBAuNsyogOHSAoCJYtM9qSFBITYelS2LfPaEs0nmbjRoiKgsuXjbYEUJ/54sXh0UeNtsQ5kZ/d\nQlr8ADWRCAEzZqj5xYy47QCklEnAcGAjcBRYLqWMEUIMFkIMdvf6vmTQIOiUZvv6/HnP7wO4S9Gi\nygksXw5JSUZbA9y5A0OGwKRJRlui8TQzZkDZstC5s9GWcOsWfPONMqVgQaOtcU5F/0YOjzt1UvOK\nWfHI7oeUcp2UsrqUspqU8t2U5+ZIKec4OfY5KeUKT4zrDV59FQIDASQgWLfO35RhoO7d4dIl+OEH\noy0BCheG55+HFStUbEqTN/jtN1i/Xs1gBQpkfbyXWb8ebtxQn30zMm1WAmvWgPXuPzBQzSdmxnxp\nDAYTHg7t24P1TUxMlPz3vQRDbXLG44+rsHtUlNGWpDBkiFqOzJ1rtCUaTzF7ttoYe+EFoy0B1Ge9\nVCn417+MtiQ989ccY8TwAJKTVexfCOjf37yhHyvaATihbFnHxxu+LUhkpDG2ZERwMDz5JHz9tYrA\nGE5ICLRtC3PmqD0BTe6nTh0YORLuvddoS7h5E9auhW7d1Kaq2VgypzzIAKw3jn5+0KePsTZlB+0A\nnNCnj2NGENKcGUHdu8PVqyozwhQMGwZ33QWnT2d9rMb89O0LH3xgtBUArFkD8fHmDP9ER8OmDUWw\nTv4ATzxh/rt/0A7AKeHhMGuWfUaQIDkZ02UEPfKImm9NEwZ6/HE4fBjuu89oSzTuIKXK6oqLM9qS\nVKKi1EKkeXOjLUnPfz455pD26edn/ti/Fe0AMiB9RpDk9zPxRpnjlAIFVEbEypWQYIZtCj8/5TXj\n4tQOtSZ3Eh0Nzz4LS5YYbQkA16+rDeBnnlEfMTNxNeEqP8QcdXiuY8fccfcP2gFkimNGEPxvfZDp\n9gK6d1eZEevXG21JCnfuqP2A8eONtkSTU2bMUMn2zz5rtCWAusG5c8ec4Z8VC0sijtvuFHND5o89\n2gFkQni42slXsT2BJdnPdHsBrVpB6dImCgMVKADt2sHCherWTZO7uHBBpfM+95xpqrujoqBqVWiY\nqYqY7/lxp4WhQyE5Wc0PuSXzxx7tALLAcUNYVQebqd4pIEBlRqxdq1YCpmD4cLVj98UXRluicZW5\nc1UW19ChRlsCqALk779Xd//uCr95ml4jD6SmfULuyfyxRzuALAgPVzv69qxeLU21CujeXe0BqCIU\nE/DQQ9CkidpJt1iMtkbjCgcOKJ2F6tWNtgSAr76C5GTzhX+io+HUT3UdnsstmT/2aAeQDV591br5\npKqDLRZzZQQ1bQoVKpgoDARqFXD8OOzaZbQlGldYsUIVl5iEqCh44AFVkmAmFiwApD/2ef+5KfZv\nRTuAbBAernb2bXm+ggsXDDQoDX5+KkNiwwa4ds1oa1Lo2hX27FHeSZM7+Ocf9T2Hmv+e5uxZ2LbN\nfOGfU9dPcfTUVYfnclPmjz3aAWQTW0YQgGT16mRTZQT16KFCt19+abQlKRQsCPXrG22FJrvExECZ\nMvDtt0ZbksrSpaokwWzhnz7jtvPDxmJYswNzW+aPPdoBZBNbRhCoMJAfQ4eaZy/goYfUUnnhQqMt\nsUNKGDAA3nrLaEs0WTFzpnq/GjfO+lgfsWiRMsck2xGAiv3vnNMrNfyTGzN/7NEOwAVsGUFqLyA5\nWZgmI0gI6N0bduyA33832poUhFAxqRkzlJavxpz8/bcKanfvrtTWTMChQ2o/ulcvoy1xZNIkW9on\n5M7MH3u0A3ABW0aQNSApTZURZK3bWbzYWDscGDYMrlwxUfcaTToWLFBqa8OHG21JKosWqRTnZ54x\n2hIbS9edYuWqJKyhH8idmT/2aAfgIs4ygsyyCqhcGVq2VGEgKbM+3ie0bq1iUzNmGG2JxhlSKtnn\nRo2gQQOjrQFU2ufixUpc9p7stTb3CTOmBOeJzB97tANwkfQZQSr/3iyrgN69Vfbl7t1GW5KCEGoV\nsGcP/Pyz0dZo0iKE0lqYPt1oS1L54QeVAWSm8E90NPy0uYzDc7k188ce7QBywKuvpm0gb566gK5d\nVb9gU20G9+mjdOXTNlrQmIPq1dUKwCQsXKjanqobLXMwaeb5lKpfdePn75/77/5BO4AcYZWLtm4I\nSymZN89iilVA8eLqHycqykR9WYoWhY8+gkqVjLZEY88ff0CXLmrJaBLi41X1b9euqumRGZi/5hgr\nl5ZMfezvr/7/c/vdP2gHkGMGDbKXiBAkJZknI6hXL/jrL1UYZio2b1ZdvTXmYM4cFf4JCjLaklRW\nr1aaVr17G22JjZWfhYClANa7/yeeMHejd1fQDsANVETDthewerU59gLatlXZfIsWGW1JGt59F15+\nWfUO1hjLrVswb55aLppoZbZokZI1adnSaEsUkZGwepUf9v/neSmSqR2AG/TpY9+gQmCxSCZOND79\nJjBQpXSvWqVSvE3D8OHw559KulRjLMuXq2WiiVI/L11Sq9aePc3R+GXnTsngIUlIu5Q6f//cnfef\nFhP8mXMvtowgG6tXS1NIRPTuDbdvK20v0/DEE1Cxoqo61RjLzJlQsyb8619GW5JKVJRKATVL9s87\n799GWmxpn0Lkndi/Fe0A3MSWEaQyBKRJGsg3bKgac5kqGyggAAYPVgLvv/5qtDX5l6Qk1bRn7FhT\nqazNn68kTWrXNtoS9f+7cZ3j3kinTnkn9m9FOwA3sTWQtymFmqGBvBBqqfrDDyaShgClDVS5Mpw8\nabQl+ZeAAKXPZKJYxv79sG8f9OtntCWKyXOupDR6zztFX87QDsADOGsgf/58slHmpNKnj3IEpmrM\nVbq0mvwff9xoS/Inf/2lMrFMthH/+eeqm6gZ2hAnJieydp9j0WJeKPpyhnYAHiJtA/k1a4XhewGV\nKkGbNuqfy1SNufz81AR06pTRluQ/Pv0UOnc2Ve7/nTtK+qFTJyhZMuvjvc3nnwZy+8hjqY9zs9xz\nVnjEAQgh2gohjgkhYoUQrzn5fSchxEEhxH4hxB4hRHNPjGsm0slFm6SB/PPPq8SbzZuNtSMdnTtD\n+/YmEi3KByQnK92fVq2gVi2jrUllzRqlF2iG8E90tGqHbElWqZ+5Xe45K9x2AEIIf2Am0A6oBfQQ\nQqT9dG0CQqWUYcDzwDx3xzUjSi7atqlmhgbyTz4JJUrAZ58Za0c6nnoKjhyBrVuNtiT/8O23qvrX\nRKmfoFao5curVsRGM2DMsVzf6N0VPLECaATESilPSinvAFGAY0RcyjhpS6YtjL2eah7CjA3kg4JU\nXvXXX5uoXSSoQoWSJXVKqC+ZOVPNtI4bVoZy7hysX2/fa8M4du6UHNlxv8NzuV3uOSs84QDKA6ft\nHp9Jec4BIcRTQohfgW9Rq4A8iRnlop9/XtUELF1qrB0OBAertfXKlXDmjNHW5H3i4tTm++DBKgvI\nJCxcqPannnvOaEvgww9FnpN7zgoh3YzBCiG6Am2llANSHvcGGkspna4zhRAtgPFSyjYZ/H4QMAig\nTJky9aOionJkV1xcHEWKFMnRue7yxhsPsmNHKdQHSSKE5JVXTvDEE+cNsUtKGDiwAf7+koiIvU6P\nMcKuoPPnadyzJycHDuR0jx6msSs75Eq7LBb8kpKwFCjgW6NwbpeU0LdvI4oXT2T69H0+t8nerkVf\nB/Pp9Iao/1n1f9us2V+8806MoXblhNatW++VUmavuYOU0q0vIBzYaPd4LDA2i3NOAqWyunb9+vVl\nTtmyZUuOz3WXnTul9PeXUn3EpQSL9PdXzxtl19SpypYDB5z/3rC/1+7dUiYnZ/hrI9/HzMhVdiUk\nSBkf73Nb7HFm186d6jM5b57v7bGyZcsWuXOnlMIvUYIl9X/W+v9qpF05Bdgjszl/eyIEtBsIEUJU\nFUIUALoDq+0PEELcL1IqpYQQDwEFgSseGNuU2IrDwNY/2NgN4WefVelsn39unA1OadDAHMIveZn5\n85UEh8lCbZ9/DoUKwdNPG2vHpEnkecmHjHD7P09KmQQMBzYCR4HlUsoYIcRgIcTglMO6AIeFEPtR\nGUPPpHiqPEv64jDB6tUQE1PMEHtKlVL2LFyo8q5NxdSp5ur+kZeQUm3+VqqkNoBNwo0bak/q6adV\nuwijiIkpxurVYK/2mRclHzLCI7deUsp1UsrqUspqUsp3U56bI6Wck/LzRCnlg1LKMClluJRyhyfG\nNTtqQ9j2wbJYJEuXVjTMnuefV/nWq1YZZoJzpFTJ4PuMiQPnabZtg8OHVeqniXR/li5V+9JGT7RT\n5xdIkXxQ5IeNX3v02tuLOFMLjY6+27C00EcfVTeCERHGjJ8hzz2nYgE6JdTzzJwJd92l0m5NRGQk\n1KkDTZoYZ0N0NJz4pabDc3lV8iEjtAPwMmnVQi3SOKE4f3+lxbZpE8TGGmODU0qUUBrAixfD1atG\nW5N3uHBBFYD0768crEnYu1d9DRpk7KJkwQLALvafV/r8uoJ2AF7G1j845ZMuBZ9+apxERP/+6oNu\ntE5ROoYNU12qTLdLnYspU0aFgF56yWhLHIiMVGUgRur+r/r+EvPm2QSy8lKfX1fQDsAHpO0fnJho\nXEbQvfeqZe7nn6viMNNQt66SKDZLL8C8gBDQtKnKADIJN27AkiXwzDNq4WcUI8adISnJmvOft/r8\nuoJ2AD4ibR9RIyUiXnjBpgpsKt58U6WFatznm29U1e+NG0Zb4kBUlPGbv9HRcHp3PYfn8lKfX1fQ\nDsBH2PoHWyUihGGrgEcegapVTbgZDBATA9OnG21F7ufjj1XntcKFjbbEgYgIYzd/pZRMmiSxWGwV\nv3ld8C0ztAPwEc4yglatMqZ/sJ+fugPbuhWOHfP9+JmyYoWKWZtqlzqXsX8//Pij0jU2UZGdGTZ/\n+77xIytXWrDXo8xvmT/2mOfTkQ9QdQG2D56UMHSoMaGgfv2UJpjpNoMHDVKGzZ5ttCW5l5kz1S6r\nGQT27Zg719jN3+hoWPxBU9S0ZxV8k/ku88ce7QB8SHg4vPzyiTQSEcaohZYpoyT5589XyTemoVw5\n6NJFNTCIjzfamlxHwI0bKp22Z0+V/28S4uP9WbxYVf4atfk7aZKt0QuoVcjLL5/It3f/oB2Az3ni\nifMpEhG2JvKrVxuTFvrCCyrt/quvfD92pgwbBtevq3QRjUv4JSZC376ma/qycWMZ4uJUVMoIJky+\nwMpVjqGfTp3U/2N+RjsAA7D1DFBYLMakhbZuDdWrm7AAt3lzlb6oi8Jc5k7Jkip8FhpqtCmpSAkr\nV5anYUNo1Mj340dHw5uj7gFpS/vMj0VfztAOwADMsiHs56dutqOj4dgxE2nbCwE7duj/UFf55ReK\nHT5suj7LmzbBn38W5sUXjRlfqX06hn7yY9GXM7QDMIi0EhFSCkOayPftqzIFv/mmgm8HzgohQEqC\nTSZhbGrGj6f2+PGQmGi0JQ7MmAElStyhWzffj711+21Wr1b/Y1byk9pnVmgHYBC2ngHGNpEvXlw5\ngc2bS3Ppkm/HzpJJk2jYr5/StNFkzsmTsG4d5554Agzo+JURp04podf27c8TFOT78V8YG4vFpviQ\n79Q+s0I7AANJ3zPAmFDQ8OGQmOjH3Lm+HTdLOnfGLykJ8xlmQmbPBj8/znXoYLQlDsyZo7537HjO\n52NHRsLxnQ84PJefc/6doR2AwaQPBeHzUNADD0D9+leZPdtk0YOQEK42bKhmEVMZZjLi4+HTT6Fz\nZ+7cc4/R1qSSkADz5sGTT0Lp0r4VnoqOTsk4krbYv974TY92AAaTPhRkTPvIp546y9mzsHKlb8fN\nirNPPgnnzpmwi42JOHLEtqNvIpYtUw2IjNj8ffmNsyQn2zbD9cavc7QDMAHOQkG+Fotr0uQKVaqY\nT4bnSuPGUKWKcU0UcgMNGqh+vy1aGG1JKlKqz9KDD/pe4DU6Gn7e7Kjupjd+naMdgEmw1QZYxeJ8\nWyHs769uILdvhwMHfDdulvj7q13EZcuMtsScXL8OFgsEBZmq5WN0NPzyizGdKCdNwiH0ozd+M0Y7\nAJNgqw2whYJWrfKtVs/zzyutFrOtAqhdWxmmSc+wYaq6ymS5/598Ymv05kvenXyJVasc0z71xm/G\naAdgImwbwgoppU/F4kqWhN69YdEizJcS+t13EBYGf/9ttCXm4eJF+PJLaNbMVHf/J0/a2hEU8WF9\n4c6dknGjSjr4Qr3xmznaAZgI24YwGCUW98orqlOY6eQh7r5bxaa++MJoS8zD3LkqO8oogZ0MmDpV\nTby+3vz98EPh0ONXb/xmjXYAJiP9hrBvxeJq1lTt8WbNMpkYZ/36qovIzJk4VPbkV5KSVHrsI49A\njRpGW5PKtWsqI7V7d9V+1Ffs+DGZ1atBV/y6hnYAJkRtCNs+yBaLZMAA3zmBUaNUy0jTJd4MGwbH\njytxmfzOt9/C2bOmU/2cOxdu3oR//9t3Y+7cKWn79GksFlvsR2/8Zg/tAEyIM7G4I0ckLVv6xgk8\n/LDKLPzkE5PdbHfrBvfco8Rl8jvt28Pq1eq7SbhzB6ZNg3/9S23X+ILoaGjVCm6eq+zwvN74zR7a\nAZiUtBXCIEhM9M1+gBBqFXDihMrANA0FC8KHH6p0pfxOQICK1dlnDRjMl1+qRcnIkb4bc9IkSEy0\nyjzril9X0Q7ApDgTiwN8th/QpQtUrqx6i5uKvn3TV83lN958E95/32grHJBSfVZq1oS2bX0zZmQk\n6fr76o1f19AOwMQMGmQT07J+yH3VPCYgAF5+WRWG/fyz98dziQsX4J13TNbL0kf884+KzR07ZrQl\nDmzdCvv2qdi/L/rQK60f2+oY1OQ/Z47e+HUFj7xVQoi2QohjQohYIcRrTn7fUwhxUAhxSAixUwhh\nnnZFJmfQICWmZY+vCsT691dy0aZbBRw5Am+8AcuXG22J71mwAOLiTKf7M2mS2p7xVeHXpEmQnGyd\n/KWe/HOI2w5ACOEPzATaAbWAHkKIWmkO+x1oKaWsA0wAfCx4nLtR+wHqg+5LxdCiRVXf4BUr4Lff\nvDuWS7RurSRM89tmsJQqDbZRI2jY0GhrUtm3DzZsUDUkvijYjo6GVavtsxOETvnMIZ5YATQCYqWU\nJ6WUd4AowCFIK6XcKaW8lvJwF2Cy9lPmxsjmMS+/DIGBMHGi98fKNkKoO+Ddu00Yn/IimzfDr7+a\n7u7//fehWDHf1aO9+tZlpMX2v6BTPnOOkG5qiAghugJtpZQDUh73BhpLKZ0mKAshRgE1rcc7+f0g\nYBBAmTJl6kdFReXIrn/++YdixYrl6FxvEhcXR5Ec1se/8caD7NhRCtuyV/LKKyd44onzXrVr8uQQ\n1q0rx5IlP3HPPb7Vdc/ILv+bNwnv1o2/Hn6YX8eO9alNmdnlTYrExlJh+XKOjxqFJYOuX762688/\ng3nuuUb06PEnAwf+nuFxnrJrzZpyfPJJCLbYv6RZs794552YHF3PiPcxKywWC/Hx8Tm2q3Xr1nul\nlA2ydbCU0q0voCswz+5xb2BGBse2Bo4Cd2fn2vXr15c54cCBA7JixYpy5cqV0mKx5Oga3mLLli05\nPnfnTin9/aUEi1TxAPV4507v2vX772qcESPcH8dVMv17vfSSlD16SGnAe+zO++hNfG1Xv35SBgVJ\nefFi5sd5wi71+bd99j3x+TfT+2ixWOSyZctkSEiIXLRoUY6vA+yR2Zy/PRECOgtUtHtcIeU5B4QQ\ndYF5QCcp5RUPjJshCQkJ+Pn58eSTT/LII49w6NAhbw7nMzIKBb2Wbtvds1Spojb3IiNNJhI3ZQos\nWWIqITSvsW6dKswwEX/+CQsXwsCBULq098cb8NJlkpNtj/NSyucvv/xCy5YteeaZZyhUqBDxPtJh\n8YQD2A2ECCGqCiEKAN2B1fYHCCEqAV8DvaWUxz0wZqY0btyYefPmMX36dH755RfCwsIYOnQoly9f\n9vbQXsdZ85ht2yRjxnh33LFjVdbllCneHcclrBP/iRM4zAx5jVu3VP3D6NFGW+LARx+p76NGeX+s\nMWPgyJ5SDs/lpY3fCRMm8OuvvxIREcHevXsJCQnxybhuOwApZRIwHNiICu8sl1LGCCEGCyEGpxw2\nHrgbmCWE2C+E2OPuuFkREBDA8OHDiY2NZdiwYURGRhISEsLkyZO5c+eOt4f3Kq++6qgYCoIPP/Ru\namiNGtC1q0pCuX7de+O4zPbtUL06rF1rtCXe48svlTiTiXR/Ll1Suj+9e0OlSt4dKzJSFYCnzfnP\nzRu/t2/fZuLEiZxIWdXNmjWLEydOMGjQIPx9WN3tkToAKeU6KWV1KWU1KeW7Kc/NkVLOSfl5gJTy\nLillWMpX9jYoPEDJkiWZNm0aBw8epEmTJvz73/+mTp067Nu3z1cmeJzwcOvNoC30IaX0emro66+r\nOiRTZV+Gh0OFCiYzysPMnKk88P/9n9GWpDJlipIN9/bKMzoahgy1WPcQUxk9OneHfq5fv867777L\nihUrAChXrhzFixf3uR35phK4Vq1arF+/nrVr11KsWDHKlSsHQFJSksGW5YyJE+3vgKy9A7ybGhoW\nprTHpkxRtUimICBAdR75/nuVIpnX2LMHfvpJpX6aZK/jyhXlb7t29b4S9aRJYLEr+AL1uTdVWnI2\nOXjwIK+88gpSSsqUKUNMTAxjDchgsyffOABQm6ft27fn559/pmzZskgpadOmDW+99ZbRpuWIiRPT\nVwl7WyvoP/9RE8Ds2d4bw2UGDoQCBdSOYF7jwAEoVQr69DHaklQ+/ljdAIwf791xoqNJ0fi3Injy\nydw3+V++fJkXXniBevXqsXDhQk6ePAlAxYoVszjT++QrB2DFmkVz+/ZtatWqReXKSko2KSmJxMRE\nI01zGcfeAaqZvDd7B4SHw2OPqX/CGze8M4bLlC6tpKKXLVMdsvIS/fvD6dNKk8ME/PWXknx++mnV\nqtlbREdDt17XUzT+c2dz9zt37vDxxx9z//3389lnn/Hiiy9y4sQJqlWrZrRpqeRLB2AlKCiIWbNm\n0a9fPwDmzJlDaGgoGzZsMNiy7JO+d4D0eu+ACRPUKmDaNO9cP0e8/z7ExKiy5bzClZRs6aAgY+2w\n48MPVac4b979R0dDi5aSsycdnV5u0fiXUrJmzRpq167NqFGjaNasGYcOHWLKlCncddddRpvnQL52\nAGmpVq0aiYmJtGvXjvbt23PMZIqLGeHr3gENG6p/xg8/VC0ATUHFiipUkldITlZtMH3dWDcTLl1S\nsf8ePaBWWrUvDzJpEiTlUo3/mJgYHnvsMTp27Ii/vz/r1q1j3bp11KxZ02jTnKIdgB3t2rXj8OHD\nfPjhh+zYsYPatWvzyiuvcM00s5xzHAvEbNkS3twP+O9/4e+/lTKxaTh5Uv0xtm412hL3WbcO/vgD\nWrY02pJUJk1SJQnevPvfuVOyerVjG7rcVPC1adMmdu/ezZQpUzh48CDt2rUz2qRM0Q4gDQULFmTU\nqFGcOHGC559/nqlTpxISEsKsWbNMnTFk6x3g2EvYW6uA0FAVdp8yRcWFTUG5cqpn8MyZRlviPjNm\nQPnypml+c+GCmoR79vRu5s+IN85gsRN6M7vMc3JyMtOmTWPp0qUADBkyhBMnTjBixAgCc0E4UjuA\nDChdujQRERHs27ePOnXqMGzYMMLCwrhw4YLRpmWIs94BK1d6r0r47bdVPNgXqqTZIjhYbZp+843q\nTZhbOX4cvvtOaXGbZBJ57z3V8/eNN7w3xpgxsGezo1Cw2at9/fz8iIqK4ttvvwUgMDCQUrkoFKkd\nQBaEhoayefNmvvrqK+rUqUOZMmUAuHnzpsGWOcfZfsCkSd4p2HngAXj2WXWzahq/OGSI6mQfEWG0\nJTknMlJN/AMHGm0JoCJrc+aoVszeUij496g7TJrk2OHLrHH/Y8eO8fTTT3Px4kWEEKxfv56FCxca\nbVaO0A4gGwgh6Ny5M0uXLkUIwcWLF6lSpQrz58832rR0OO8lLPnwQ+/sB7z5prozfPddz187R1St\nqqrVIiNrzrhNAAAgAElEQVSVYbmRt95SewBlyxptCaDu+gMC1HvtDSIjYfLHAQ7PmTHuf+3aNV55\n5RVq167Nxo0bOXDgAADFixdP17s7t6AdQA7w9/fnySefJDzl03n9+nWSTSRGNmiQvW6YtYuYd/YD\n7r9fRV3mzIHYWM9fP0e89pqaRN3sdWEYRYpAmzZGWwGobl9LlqjGQOXLe/76qrcv2N/5g/r8miX0\nk5SUxKxZswgJCWHq1Kk8//zznDhxgkcffdRo09xGO4AcUKpUKebOnUuNlN2w559/nvr167PVRNkn\nEydCixaOz61c6Z1Q0NtvQ8GCSivIFDRrpuQhChY02hLXkFJVWH31ldGWpDJ2LNx1l/dCMa+9ZkkR\ncrVN/maSeli/fj1169Zl2LBhqRpiERERlPaF/rUP0A7AA/To0YPr16/TunVrOnfuTKxJboU/+MDa\nSxisWiqTJnl+U7hsWSUJ/OWXSrbGFMTHqxjC4cNGW5J9tm9Xf0STyK1u2QIbNyr5jxIlPH/9ESMT\n2LbNMXXZLFIPx48fp23btjz++OMkJiaycuVKNm/eTGhoqNGmeRTtADxAt27dOHr0KO+88w7fffcd\ntWrVYuTIkYbXD9j2A8C2KYxXpKNHjYIyZdR3U0Rebt9WcQRTNTDIgpkz1e12jx5GW4KUKpJWsaJ3\nWhBHRsL0ydYKZ/NJPCckJLBnzx4mT55MTEwMnTp1yrVx/szQDsBDBAcH85///IcTJ07Qt29fJk+e\nzP3338/06dMN1Rey7QdY77QEUqoIiSedQJEiKhS0Y0daAS+DuOsulbS+ZAlcvWq0NVlz7hx8/bVK\ntSlUyGhriIqCn39WBX+eVqKIjFSfPynTx/2N3PSdMmUKQ1M624eGhnL69GlefvllCmTQfzkvoB2A\nhylXrhxz585l37591KtXj5deeomGDRsaWkRmk4527B8wZIhnM4P691dFQmPGgClq5oYNg4QE+Pxz\noy3JmshIJf8wZIjRlhAfrz4vDz3keRHS6GgYPCS9vr9RcX9rb1xQqp1nz55NvWELDg72vUE+RjsA\nLxEaGsr//vc/1qxZQ//+/QkICEBKyZkzZwyxJ710tPB4pXBAgBrn2DFrVbLBhIZC8+YqDmaxZH28\nkTz0kJoFTaAU+dFHcOaMip75eXiGmDQJpMXxzt+ouP/PP//Miy++yPr16wH473//y6pVq3JFBa+n\n0A7Aiwgh6NChAy+mCHpt2LCBPn368N133xlij5KOBvtNN09nBnXsqBpXjR9vEomI4cOVRISputk7\noWNHtWtvMGfOqMm4Wzd4+GHPXnvMGPV5s5/8jZB4Pn36NL169aJx48acO3eOW7duAfi0FaNZ0A7A\nhzRp0oR+/frRqlUrAA4cOEB8fLzPxg8PV41cbJtZtsygiIiqHhlDCJg6VbWO9KZsQLZ5+mm1MWGS\noiqnfPEFXL5stBWA2vhNTva8vMeciMoplb62mw8h1OfRV3H/uLg43njjDapXr86KFSt4/fXXWbRo\nEZ07d/aNASZEOwAfctddd9G7d28KFCjA7du3efzxx6levToLFy7E4qMQhVU0Lm1mUFRUJY/tBzz4\noAq/R0bC/v2euWaOsTq7y5fNuQo4cACeew4WLDDaEnbtgsWLYeRIqFLFc9eNjoblyyqnPLJl/PhK\n5C05OZnPPvuMkJAQ3nnnHTp37syxY8d49913KWSCDXcj0Q7AIAoWLEhUVBTlypWjT58+NG7cmO3b\nt/tkbGeVwuDZTmJvvQUlS8JLL5kgLfTGDSURYRrVOjtmzlQidilNiYwiOVm9V2XLquIvTxEdrT5X\nzjJ+fDH5nzx5kgYNGtC/f3+qVq1KdHQ0ixcvTu0CmN/RDsBAHn74YX766ScWLlzIhQsXaNGiBV27\nduW3337z+tiqUtjxn9KTncTuukspSG7frjo1GkrRokof6NNPVYqLWbh2Td1yP/us8pYGMncu7N6t\nakSKFPHMNaOj4eEWyRw54ngH0KKF9zd976ToQJUrV45ixYoRFRXFjz/+SJMmTbw7cC5DOwCD8fPz\no1evXhw7dowJEyawYcMGHnjgAUaNGsV1L1eEqkph6yNrJzHJa6955vrPP6+SW0aNMkH/4GHDVIVt\nim67KZg/Xzkkb1RaucClS+quv3VrVTrhCax3/slJfqRV+PT2XvfUqVOpU6cOt27dIjg4mB9++IFn\nnnkmTxZyuYt2ACahUKFCjBs3jhMnTtCnTx8++eQTr29Opa8UVmzb5pnMIH9/df1z50ywIfzww1Cn\njtKuNjwmlUJsrEpTrVfPUDNGj4abN+0/C+4RHa0amR05Yv+s9KrC5+3bt4mLiwOgdu3atGzZktu3\nb3t+oLyGtRDCjF/169eXOWXLli05PtebZNeu/fv3y59++klKKeW1a9fkmjVrpMVi8YpNERFSQrJU\nM6OUYJFgka++6pnrDx0qpZ+flLt3u36uR9/HOXOkFELKmBi3L+Uxu27d8sx1UnDVrq1b1Xv++uue\ns6Fp88SUz5BM/RIiWUZEeG4MKxaLRS5dulRWrVpVjhw50uXzc/s84Qxgj8zmHKtXACYlNDSURo0a\nATB79mw6duzIiRMnvDLWoEHQvfvplEe2TWFPCce9957SCRo40OAK4V694MQJ73Y0zy7WIgkDFUvv\n3FGFx1WqKME3TzBmDOzc4ZhPLwS88soJj2/6bt++nSZNmtCjRw+KFStG27ZtPTtAPkA7gFzAqFGj\n2LhxI9WrVwdgzpw5nDp1yqNjvPDC73ZyEfZOwP1wUPHiMG2aSgmdNs1NQ92hcGFbpa2RYaDff1fF\naYsXG2cDquL36FGYPt0z8kNjxlgTrRyTC+bMgSeeOO/+ACkcO3aMp556ihYtWnD27Fnmz5/P3r17\naWOSHgq5CY84ACFEWyHEMSFErBAi3RaiEKKmECJaCHFbCDHKE2PmJwIDA3nkkUcAuHTpEiNHjqRG\njRqMGjWKqx4UOkuvGWQtFHPfCXTpAh06qL2AP/5w71pukZgITz0F779vnA2zZysHlFIQaAQxMUq8\nr1s39b64y5gxMl2hF6jPk6fu/C9dusSwYcN48MEH2bRpE++++y7Hjx+nb9+++bKK1xO47QCEEP7A\nTKAdUAvoIYRIu8a+CrwEfOTuePmd0qVLc+zYMXr27Mknn3xCtWrV+Oijj1LL2d3F5gTAthJQTsAd\n9VAh1P4rqF7nht2ABwbaegUYodKakKDSUZ96yjsttrJBUpLK0Cpa1PaeuENkpH2JhXcau1y6dImQ\nkBAiIiJ44YUXiI2N5fXXX8/3hVzu4okVQCMgVkp5Ukp5B4gCOtkfIKW8JKXcDRini5yHqFChAp99\n9hn79++nSZMmjB49mpo1a7J48WKPVBQ7dwLuS0hXrqyuvXEjzJvnrpVuMHw4nD0Lq1b5fuyoKCVP\nbWDq55QpSup5+nRwt7GVVdrZFvZRnt0Tk39ycjI7d+4E1I3Pm2++SUxMDDNnzswzHbmMRkg3b8WE\nEF2BtlLKASmPewONpZTDnRz7FhAnpcxwJSCEGAQMAihTpkz9qKioHNkVFxdHEU9VtHgQb9i1d+9e\nIiIiOHHiBCEhIQwZMoR6LqYWOrNrxIhQDh4sge2uTiKE5JVXTuQ4pmuxwKhRofz6a1E++2wPZctm\nvnLxyvuYnEyTnj1JKFeOA5Mn5+gSObXroaFD8U9IYPdnn3km59JFu06fDmbAgAY0bHiNCRMOu2XC\nmjXl+GRyCDhU+Urq1LnOtGkHXLLLGUuWLOHTTz9l/vz5VKxYMeeGZkJenCdat269V0rZIFsHZzdd\nKKMvoCswz+5xb2BGBse+BYzK7rXzcxqoqyQnJ8uFCxfKSpUqyRdffNHl853ZtXOnlIGB9qmh6rsQ\n0q2Uvt9/l7JIESlbt5YyOdl1uzzCxInqhR06lKPTc2zXuXM5y4fNJpnZlZQkZbNmUpYoocxwh4gI\nlVFr/7kA9XnZudM1u+w5ePCg3Lt3r5RSyr/++ksuXbpUJmf1IXGDvDhP4OM00LOAvXuukPKcxofY\nVxS/8847AGzbto0BAwbkuDVleDj88EParEn3O4pVqQKffKJ6zs6albNruE3//moXtEwZ345brhw0\nyN7Nmaf58EP48Uel1lquXM6vo8I+MmUfx5YwUKuW+rzkpNDr9OnT9O/fn7CwMF5NiT/efffddO/e\nHT9PNyXQpOKJv+xuIEQIUVUIUQDoDpihKWC+JCgoiGLFigFw6NAhtm7dSlBKTz+Zg3BfeLiK1/v5\nOcYKpJRuOYEBA6BtW5Vd5KXyhsy5+27VtOCee3wz3sWL8MgjsG+fb8ZLw549KgOrWzfo3Tvn11GT\nvyXdJr6fn/qcuDr5X716ldGjRxMSEsKiRYt4+eWXWb58ec4N1LiE2w5ASpkEDAc2AkeB5VLKGCHE\nYCGE2h4SoqwQ4gzwb2CcEOKMEKKYu2NrMmfYsGEcOXKE4OBgbt++TXh4ODNmzEgVysou1j4Cthsx\na2/hnDsBIdSEERQE3burHu4+R0pYscLapcS7zJsH33+vlD99zM2bSm+ubFmIiMj51sOYMdYMLscN\nXz8/13X94+Pj+eCDD7jvvvv4+OOP6d69O8ePH+fjjz+mpMHCePkJj6ytpJTrpJTVpZTVpJTvpjw3\nR0o5J+XnC1LKClLKYlLKEik//+OJsTWZY21ofeXKFYKCgnjxxRdzlDE0aJDqq+LJcFD58vDZZ/DL\nL56VIM42QqhqqNde825ealKSqoZq0wZq1vTeOBnw8stKdmjhQqXSmhNUkZf1b+QY9tmxI/u5/klJ\nScydO5eQkBDGjh3Lww8/zIEDB5g/f76WaDYAHVzLJ9x7771s2bKF9evXU7x4cXr16kW9evX49ttv\nsx0a8kY4qFMnlZU5eTJ8+63r57vNsGGqifGmTd4bY/Vq1WtxeLrEOK/z9dfqPRszJud1ZyrPP/3N\nQmCg62GfW7duMW7cOKpUqcL27dtZs2YNderUyZlhGrfRDiAfIYSgbdu27N27l6VLl3Lz5k06dOhA\nixYtOHToULaukVk46IUXclYx/OGHqn/7c8+p9Hyf0q2b2gfwREVURsycCZUqeabk1gVOnlR73Q0a\nqP3unGAN+6TN82/RIvsbvrt27WLChAkkJSVRpEgRfv75Z3bs2EHz5s1zZpTGY2gHkA/x8/Oje/fu\nHD16lNmzZxMbG8tLL73EiBEjsnV+RuEga8Vwr16u2RMUpJrGxMcrPXqfCsYFBakd6TVr4M8/PX99\nKVXD9zfftG++4HUSEpT8hhCwfDmkRAJdolcve3kHW9gnu5O/dWV58eJFDh06lNroqHLlylqb3yRo\nB5CPCQwMZPDgwfz2228MGjQoVW/oxo0bnDx5MtNzreGgwED7f2TlBBYvdl1FtEYNtbL44QfP9CJw\nicGDlQGnT2d9rKsIASNGKO0FHyGlimzt3w+LFqlumK4yZoy9Vp3tPQ4MzLqhS2xsLL169eLtlGVH\nx44dWbRoETVq1HDdEI1X0Q5AQ6FChejRowcdUkIUU6ZMoWbNmpzOYkK01gm0aAHqLtE9FdE+fVSY\n/JNPYMkSl19GzqlUSamjNWvm2ev+84/a5U5I8Ox1s2DePPj8c5X2+fjjrp+fXtUze2Gf06dPM3Dg\nQGrWrMnXX3+dmr8vhEhNRtCYC+0ANOno378/06dPTy2//+abbzIsJrM6gZ49bVIAtnCQdDkc9Mkn\naqLp39/HKfNCqIk6NtZz11ywQL2QbO6veIJffy3K8OHw6KMq6uQqT/e47UTVU9CzZ8aT/8WLFxkx\nYgT3338/CxYsYOjQoZw8eZLx48fn9GVofEV2S4aN+NJSEL4jI7vOnTsnAwICZPHixeXbb78tr1+/\nnuE1Xn01vWwEWGSLFs7lATLi4kUpK1SQsnJlKVeu3OHaC3GHpk2lbNQoW4dm+T5aLFLWrCllw4bu\n25VNTp+W8u67b8nKlaW8fNm1c3fulLJFC9t7Zt/NK6POcFeuXJFjxoyRhQoVkv7+/nLAgAHyjz/+\ncHpsbvvcG01ukoLQ5GHKlSvHnj17aN26NW+++SZVq1bl/fffT+2/ao9jPwHrSkCwbZtqfZvdNNHS\npVX64oULMH78g74rEuvRQ8lk7t7t/rU2b4Zff/VZ6mdcHDzxBCQk+LN2LZQqlf1zIyOheXPJtm22\n98xKRqqe586do2rVqkyaNIknn3ySo0ePMnfuXCpVquT2a9H4Du0ANFkSGhrKN998w549e2jatCmv\nv/46VatW5aOPPiI+Pt7h2IkTVSaP/SQCYLG4VivQsCHMnw8HD5bgueeUiqjX6dMHihRRaZvuMnOm\nmoWfftr9a2VBcrL6mx88COPHH6F27eyfa9X1UX9fx/esZ0/HyT8+Pp7//e9/gKorGT16NAcPHmTx\n4sWEhIS4/0I0Pkc7AE22qV+/PmvXriU6OpqHHnqI0aNHc9999zEzzYS5aFHafgKQk1qB7t1h4MCT\nREV5rmdtphQrppxAVJStZ29OSExUBQ39+6s0Uy8iJYwcqWrNpkyBxo2z3yHOJu0AaSf/V19V76M9\nb731Fu3bt+fChQsAjBs3jtqueBuN6dAOQOMyTZo0YePGjWzfvp1atWqxz263NikliX/iRKvujC2L\nxH5zOCwMoqOzHqtHjz8ZNEilHs6e7fGXkp6hQ5UwkTv6QIGBsGsX/Pe/nrMrA957T6l7vvwyvPhi\n9s6JjIQKle44yfFXe+EREer9u3nzJh999BHRKW/UiBEj2LRpE2XLlvX8C9EYgnYAmhzTvHlzNm/e\nnLoC+Pnnn7nvvvtSHcKgQUoCx1E6Qv184ICkWbOsVwNCqGhK+/Yqt93rfdQffFBl7fTvn7Pz79yB\n69eV4V5OfZwzB8aNU4V3H3+cvXN69VJ3/WdPB6Y8Y3tv/PzUNZ99No6JEydSpUoVRo8ezerVSty3\nfPnyPPzwwx5+FRoj0Q5A4zYFCxYE1N1+nTp1uP/++wGIiYnh2Wfj2LHDWisA9pvDUiqBsZYtM18N\nBATAl19Cy5bQty988403Xw1Qu7aawHMiELd8uVK5O3rU83bZsWyZWqy0b69KDbKSzI+OhodbJLF4\nsf1qzDb5t2gBGzfe5MqV96lSpQqvvfYaDz30ED/++CPvv/++116Hxli0A9B4jIYNG/Ltt99StGhR\nLBYLXbt2pXLlymza9A6rVl13ujkMsG1b1quB4GAV527YEJ55Btav99rLUIwfD507u37ezJlQoYKq\nLPYSUVFqg7Z5c+VvAgMzP37MGFXjtmN7eikKVaicwKOPvsszz1Ti9ddfp1GjRkRHR7Nx40aaNm3q\npVehMQPaAWi8gp+fH59//jlNmzbljTfeoHLlylSpMo6PP76BTfU37WqATFcDRYvCunXqBv3JJ73c\n071AAbUPcOxY9s/Zu1fF/ocNy/qWPIcsWaIm/2bN1N+iUKGMj42OhhYtLEyaZN+9y+aAQ0Nh6tQ9\nfPZZGcaNG0d4eDg//fQT69ato0mTJl6xX2MutAPQeI0mTZqwZs0a9u3bx2OPPcZ7773HG2+UpUuX\nkQwZ8g/2MgMKmeVq4K67VF+VsDAldrZ0qZeMHzhQ3Vq70rNy5kwoXFjFqbzAggWqm1eLFmryz6xn\nuPWuf/t2Z6Jrkt69z7J/P/TufT9PPfUU+/btY+3atTRq1MgrtmvMiXYAGq8TFhbG8uXLiYmJoUuX\nLkydOpW5c++mceNP7bpTOV8NxMSkbxxXsiT8739qguvZE+bO9YLRZcooqej581WVVVb884/yRr17\nQ/HiHjVFSiWZ3bev0vRfu1b5GWfExBTL4K5fOVohJOXKvcXx410AKFGiBF988QVhYWEetVmTOxDS\nm52Q3KRBgwZyz549OTp369attMppBwwvou2CU6dOMXXqVL7//ntmzdrHuHEBKVWoYAtRyNTvr77q\n57QaNT5erQI2bIDXX4cJEzwceYmOhqZNVf7p4MEOv3L694qJUTNzlSoZXjIxMZEzZ85w69atbJkg\nJVy7BjduqHBPqVLOWzrevg1//w0JCY5pnXZXomBBuOsuAdwmICAAfx/KU9+6dSu1N7WZyM12BQUF\nUaFCBQLTbAIJIfZKKRtkZ5yAnJuo0eSMKlWqMHnyZCwWC35+fmzceIvSpf/DjRuTAH/S5qZPmqQ2\nO8eOdWw9WKiQ2hgeOlTlw//2m7ph99j/c5Mm8O670Lp19o5/8MEsDzlz5gxFixalSpUqWWriJyfD\n779DwYJKsLRCBeeT/5kzcPOmpHBhQeHC9n87282dv/8ZqlcvSeGMlg5e5saNGxQtWtSQsTMjt9ol\npeTKlSucOXOGqjnR+05Bh4A0hmGVC/b392fGjFAiIo6kSRcFa/ji1CnJCy9IqlZ1lJMIDFSPJ05U\nqZGtWnlQ1l8ItbTIKqNn3TqVmnT5cpaXvHXrFnfffXeWk/+tWyqT9Pp1qFhRfaU95fJlJf9w4YKj\ncqfC2rA9nnvv/YfQ0PKGTf4azyOE4O677872SjIjtAPQGE5gYCB9+vRh0KA6/PAD9O69A0jGsceA\nmthOnVKFTPaVxEIo6YKvvoIjR6BePRUW8hi7dqly24yYNk21SCtRIluXy2ryv35dTf5JSVC9utqO\nsCcuDmJiJH/8IblzJ/WqKd9tUs5ly96mXr1C3Htv8VRnq8k7eKKrmv5UaEzHggUPs3z5BcqX/w2w\nkLbZDEgOHJA0bYrDiqBzZ9izB+69VzVC+c9/sJsg3eCrr5TgjrOGxcePw8aNao8gq4T8LLBYVFfK\n2FiVhfrAA0qeyIr1jv/XX2WaHjPWTV418RcrZqFmTUGFCkG69aImU7QD0JiSbt0qcObM/XzyyW6q\nV7+AbYJztiKwhYaqV1c37P36qX2BRo1Ua0S3GDJEzc7OpExnz1YT/8CBbg1x86ZavVy6pO74H3hA\nxf7BOvHb3/EL0mb3gEoLrVQpnurV/TNNEbWnSpUqHD582OE5i8VCly5dqFGjBqGhoTzyyCOp/XzT\nsnXrVho0yNZ+o8aEaAegMTX16t3i2LFy7NzpR0iIVXo6bccqUvYIICRE3awPGKAKxS5eVNXDb7yh\nsoZyxH33qSVFRITjkuLmTdV7sWtXyKFAWnKyuus/elT9XL26ivfHx8MffyhZoj/+cLaSsf0NgoOh\nZk1BzZqC4GDP6Gb37duXo0ePcuDAATp16sQg+913TZ5BZwFpcgXh4XD8eGEiI+G995L54w/rvYtj\nxlBsrCQ2FubMEYSFQdu2qrHMO++oDKGJE1XfF5cjI8OGKSfw9ddKpxqU7PPQodCpk8uvR0q1sNiz\nR/0cGKg0jxIT1WLDYkmbnp225abAz0+FiuwjT8nJwfj7qz2SKVNcNgtQm/MdO3ZMfRweHs6UHFxs\n4sSJLFy4EFC1IHPmzKFIkSKsWrWKcePG4e/vT1JSEjNmzKBVq1a8/fbbLF26lKAgFbrasmULJbK5\nr6LJGXoFoMlVDBoEp075ExEh0khKpBU5k+zfL5k/X7Jhg7pBv3FDFY41aaKKqVwqgXnsMeWFbt60\nPVeihIozNW7s0mu4dk2Fe/7+Wzkif39195+QIElKknaTf9pQj5r8/fxUqmvhwm5vO2SLGTNmODiE\n7LB+/XoWLlzIzp07OXToEMnJyUyYMAGA8ePHExkZyf79+zlw4AAPPfQQV69eZfLkyezbt4/9+/ez\nbds2imQ3jqXJMXoFoMmVDBqkvqKjBUOH2sf507c1BElKDxNAsGePap9YvryqMejSxRZvzxA/P/jx\nx9SlQ+HYWOUM2rZVM3gW3LypdHzuu0/9rDKXJMnJ4BjLT7s0sb2e4GCoXDlzCYgbNxI8mtc+adIk\njh49yubNm1067/vvv6d79+4US9nF7tevH2PHjgXgX//6F6+88gpdunShXbt21K5dm+TkZO6//376\n9OnDo48+SocOHUyZn5/X8MgKQAjRVghxTAgRK4R4zcnvhRBiWsrvDwohHvLEuBpNeDjs26fC8w88\nAM7SIdPeSVvvsM+eVSuC4GBVYVu7dhYtK4VQt+oHD1J5yRIlrp9Jw2KLRWnylCyphOwGDZIkJiq7\npCRl8k+9eBrbbc8HBamJ/8EHM5/8Pc306dNZsmQJ69ato1BmqnMuMnnyZObOnUuBAgXo1q0bc+fO\nxd/fn127djF8+HDOnDlD/fr1OXjwoMfG1DjH7RWAEMIfmAk8ApwBdgshVkspj9gd1g4ISflqDMxO\n+a7ReATbikDd1e/bJ/jzT2uYx7kGvvV5KeHKFcGVK2oj+cUXLRQr5s8996jNWCFUtOfaNRBXr1Pi\nRhLXmIsoWpwSzQpx+7ZaQVy7pnL3/f3hyhULN+MTQaZtCpO+OU56BIUKqRDP3Xf7dtK3EhERQWRk\nJJs3b6ZkyZIun9+mTRteffVVRowYQZEiRfjiiy945JFHADh27Bh16tShTp06xMXFsXv3brp3705c\nXBwtW7akZcuWREdHc/jwYerWrevpl6axwxMhoEZArJTyJIAQIgroBNg7gE7AAqmEh3YJIUoIIcpJ\nKc97YHyNJpXwcFvDmOhopaC5a5dIkwrqLEwE1hXCnTt+/PWXzKAtcMmUL+AGsN+xYtnxWgXsfk47\nTnqMmvTbtGlDQIBtKoiOjmbIkCFUrlw5ddIuWLAgP/30k9PzDx48SIUKFRyuN3/+fA4ePEh4eDgA\noaGhjBs3DoDXXnuNEydOEBAQQIkSJfj000/5+++/6dKlCwkJCVgsFh566CE656Qfg8Yl3BaDE0J0\nBdpKKQekPO4NNJZSDrc7Zi3wgZRyR8rjTcAYKWU6pTchxCBgEECZMmXqR0VF5ciuuLg4U24iabtc\nw1N2xcQUY+PGMpw6VYizZ4O5ejWjoL83Cqds/2Pr1/9KqVI1AQgIkAQGWihQwELx4okeSeFMTk72\nqchbdtF2uUZ27YqNjeXvv/92eK5169a5VwxOShkJRIJSA82pQqVW3XSNvG5Xq1Yqk9OKLVSkQjyB\ngd5Vh7gAABUSSURBVHDiRFZXyehmKSunIShbVmUiBQZCiRLqcZEiAts2nGf6B+dWcTOjyO12BQUF\nUa9evRyP4wkHcBaoaPe4Qspzrh6j0fgM+1CRlbROITXun/qzSP35woV4goIKUaIEDnsAac8LC1M6\nRSmREI4ehZSWyRqN4XjCAewGQoQQVVGTenfg2TTHrAaGp+wPNAb+1vF/jdlw5hQyYuvWn025YtJo\nXMFtByClTBJCDAc2osTcP5NSxgghBqf8fg6wDngciAXigX7ujqvRaDQa9/DIHoCUch1qkrd/bo7d\nzxIYlvY8jUaj0RiHloLQaDSafIp2ABpNPsaZHDTARx99RI0aNfDz82Pt2rUZnq/loHM32gFoNJp0\ntGzZknXr1tHC1qNTkwfRDkCj0aSjYcOGVKtWza1rTJw4kdq1a1O7dm2GDBlCXFwcAKtWraJOnTqE\nhYVRu3Zttm7dCsDbb79NzZo1CQsLo169ely/ft3dl6HJAtMVgmk0+QpnqaRPP636DMTHqx4EaXnu\nOfX111+qGY0dwcnJMHy4alJvIPZy0EWLFuXZZ59lwoQJTJw4MVUOOjw8nOTkZG7evJkqB33+/HmC\ng4O5ceMGwcHBhr6G/IBeAWg0Go9jLwcthKBfv358//33gE0O+sMPP+To0aMUK1aM4sWLp8pBz507\nl7i4OAd9Io130H9hjcZIUsIfTilUKPPflyqV7vcJJpU2sGfy5MkcOnSIzZs3061bN/79738zcOBA\ndu3axY8//sjmzZupX78+GzZs0GqgXkavADQajcdp06YNy5Yt48aNG0gpncpBjxgxgl69erF7925u\n3LjB5cuXadmyJW+//Ta1a9d2mp2k8Sx6BaDR5HPSykEfOnSIefPmMXXqVC5fvsxzzz1HUFAQR44c\nSe3wZY+Wg869aAeg0eRjTp065fT50aNHM3r06CzPb9WqFXfu3HH6uzFjxjBmzBhAqVtaZb2/yUBw\nKaN+AxrvoUNAGo1Gk0/RDkCj0WjyKdoBaDQaTT5FOwCNRqPJp2gHoNFoNPkU7QA0Go0mn6IdgEaj\n0eRTtAPQaPIxVapUoWbNmoSGhlK7dm2ioqKcHudM9//w4cNUqVIlyzFOnTqVreNyghAiVWXUF8TH\nx9OgQQNu3rwJwMWLF3n00UepXr06oaGhGdYyzJ07l7p161KnTh3q1q3LokWLUn83f/58SpQoQVhY\nGGFhYTz11FOpv5s9ezbvvfee116PLgTTaPI5K1asoHbt2uzbt4+mTZvSpk0bSpUqZbRZ6UhOTsbf\n399QG6ZPn06XLl0oXLgwAGPHjqVFixZ899137Nixg169enH8+HGEEA7nhYSEsHXrVkqWLMmZM2cI\nCwujefPmqY6xTZs2rFixIvX4GzduADBgwABq1qzJ8OHDnVZhu4teAWg0BtKqVassvz766COH4+fP\nnw/AX3/9le7Yxx9/nFbOJKazQb169ShatCi///67y+eeOnXKwWmkfQwwcuTI1Lvg7du3pz6/bt06\nmjVrRv369QkPD2fXrl2AWnXUrVuXfv36ERYWxvr167Ntz4YNG6hXrx5169bl//7v/4iNjQWUDlF4\neHjqimfatGlAxj0K0hIZGcmzzz6b+nj58uUMHjwYgObNm1OwYEH27NmT7rxWrVpRsmRJACpUqEC5\ncuU4c+ZMlq8jMDCQRx99lGXLlmX7tbuCXgFoNBoAtmzZwq1btwgJCXH6+yNHjhAWFpb6+NatW9m+\n9tWrVwkNDeXjjz9m69at9OjRg99++40zZ84wYcIENm7cSLFixYiJiaFdu3b8+eefAMTExBAREZGq\nKZQdLl26RO/evfnhhx+oVasWn376KT179uSnn35i1qxZdOzYkbFjxwKkjuOsR0FaTp8+zc2bN6lc\nuTIAV65cQUrp4OgqVarE6dOnadiwYYb2bd26levXr1O/fn2H5+rWrUuJEiUYM2aMQye28PBw1q1b\nx8CBA7P9N8gu2gFoNAaS0Z1mdo4vVapUuvNv5EAOumvXrgQFBVGsWDG++uorSpQo4fS4WrVqOdzd\nHj58mA4dOmRrjAIFCtCrVy9A3Q0HBwdz7NgxduzYwW+//eYw4SUlJXHx4kVAhU5cmfxBaQqFhoZS\nq1YtAPr168fQoUO5ceMGLVq04NVXXyU+Pp7WrVun7mtYexR06dKFdu3aUbt27XTXPXPmDGXKlHHJ\nlrQcOXKEPn36sHTp0tSGNx06dOCZZ54hODiYffv20a5dO9auXZtqW9myZbO1WsgJOgSk0eRzVqxY\nwf79+9m2bVuqZPNTTz2VuilpjUdnRkBAABaLJfVxdlcHUkratm3L/v37U7/OnTuXOtFaBeQ8RZcu\nXdi+fTvVqlXjgw8+SL2rnjx5MnPnzqVAgQJ069aNuXPnpjs3ODjY4XXdfffdgArFWfnzzz+pWLGi\n07FPnDjB448/TkREBM2bN099vlSpUqnOoF69ejRr1szB0d66dctr3dG0A9BoNOn45ptvUifk7Kwo\nypYtS2JiYmqsfcmSJQ6/v3PnTupz27dvJyEhgZo1a/Loo4+yYcMGYmJiUo/dvXu3W7Y3adKEAwcO\n8OuvvwLwxRdfpO5vxMbGUrZsWZ577jnefPNN9u7dCzjvUZCWGjVqcP78eW7fvp36XLdu3ZgzZw4A\nO3bsICEhwSG0Y+XkyZM89thjTJs2jXbt2jn87uzZs6k///HHH+zatcthBXL06FFCQ0Pd+ItkjA4B\naTQatwkICGDq1Kk88sgj3HPPPbRv397h9yVLlmT//v1MmjQJKSVLly6lQIEChISEsGjRIvr3709C\nQgJ37tyhWbNmmcbQ01KjRo3UrJvChQtz7NgxFi5cyLPPPktSUhL33HNPatrl8uXLWbx4MQUKFEAI\nwcSJEwHnPQrSEhwcTOvWrdm6dSuPPfYYAB988AG9evXiiy++IDg4mIULF+Lnp+6rBwwYQMeOHenY\nsSNjxozhypUrjB8/nvHjxwMwceJEHnvsMWbOnMmqVatSezK89957DhP+xo0beffdd7P993AJKaVp\nv+rXry9zypYtW3J8rjfRdrlGXrPryJEjnjUkDf/8849Xr59T8opdP/74o+zUqZOXrLFhtevo0aOy\nRYsWGR7n7PME7JHZnGN1CEij0WiySdOmTenQoYPTLCFvcPr0aWbPnu2167sVAhJClASWAVWAU8DT\nUsprTo77DOgAXJJSpt9e12g0mlzCgAEDfDaWdVPeW7i7AngN2CSlDAE2pTx2xnygrZtjaTQajcaD\nuOsAOgFfpPz8BfCks4OklNuAq26OpdFoNBoP4q4DKCOlPJ/y8wXAvSoJjUaj0fgMoTaNMzlAiO+B\nsk5+9R/gCyllCbtjr0kp78rgOlWAtVntAQghBgGDAMqUKVM/I3XCrIiLi/N4EYkn0Ha5Rl6zq3jx\n4tx///1esEhhBsE0Z2i7XCO7dsXGxvL33387PNe6deu9UsoGGZziSHbThZx9AceAcik/lwOOZXJs\nFeCwK9fXaaC+Q9vlGjoN1DW0Xa6RXbuMTgNdDfRN+bkvsMrN62k0Gh+i+wG4Rtp+AK1ateK+++5L\nlc34/PPPMzx3woQJVKtWjWrVqjFhwoTU572t+Z8Z7lYCfwAsF0L0B/4AngYQQtwLzJNSPp7yeCnQ\nCiglhDgDvCmlTF9qp9FofI7uB5B90vYDAJg2bVqWonjbtm3jyy+/5PDhwwA0btyYli1b0qJFC69r\n/meGWysAKeUVKeX/SSlDpJRtpJRXU54/Z538Ux73kFKWk1IGSikr6Mlfo1G0mt+K+fvnA5CYnEir\n+a1YdFDJFsQnxtNqfiuWHVZa8H/f+ptW81vx9dGvAfgr/i9azW/FmmNrALgQd4HHlz/OhtgNObJF\n9wNwvR9Adlm2bBl9+vQhODiY4OBg+vTpk6rx723N/8zQWkAajQbQ/QBc7QdgZeTIkYwdO5bQ0FAm\nTpxI+fLl0537559/OjTqqVSpEtu2bUt97E3N/8zQDkCjMZCtz21N/TnQP9DhcaHAQg6PiwcVd3hc\nqlAph8dli5Rl3dPrdD8AH/YDWLhwIRUrViQ5OZn333+fZ555hh07drhkL3hX8z8ztBaQRpPP0f0A\nctYPAEjV/vf392fEiBHs2rXL4e9gpVKlSvzxxx+pj9P2DfCm5n9maAeg0WjSofsBZN0PwH6lArB0\n6VLq1KmTKgdtT7du3ViwYAEJCQkkJCSwYMECnn766dTfe1PzPzN0CEij0bhNfuwHcPv2bdq3b8+d\nO3eQUlK+fHmHNFr7fgCtWrWic+fOPPjggwD06dOHli1bph7rVc3/zMhuwYARX7oQzHdou1xDF4K5\nRl6xyxv9AJxp/ueWQjCNRqPJN3ijH4C3Nf8zQ4eANBqNxgU83Q/A25r/maFXABqNRpNP0Q5Ao9Fo\n8inaAWg0Gk0+RTsAjUajyadoB6DR5GO0HLRrpJWDHjZsGHXr1iUsLIyGDRuyadOmDM/Ni3LQGo3G\nB0RHw9at0KoVuCiN8//tnX1oXWcdxz9fbELqHDG16Lplmy2UYQWl7RhLHVqbKWthav8oTGROHISB\niqNqaZGVkUGpghYEFeYcrFJcC8YXRoddG8Oo64axNLudcesqE1O6JqYhcX/Uav35x3lSj8l9Oeee\nl6S9vw9c7jnnefvmd56c55znnPO9DXE76OTMtYPes2cPnZ2dAIyMjNDb28vExMTVF9NmuS7toB3H\nKZ4TJ6C3Fx57LPo+caKYdtwOOr0d9OzBH2B6enregX8Wt4N2HKcphobg8mW4ciX6HhrK/yoA3A66\nWTvo3bt3c+DAAaamphgYGKg6CLgdtOM4TbFxI7S3Rwf/9vZoPU/cDrp5O2iA/v5++vv7GRwcZMeO\nHRw/fpz29vZUmhfKDtoHAMdZ5PT0wLFjxd8DiLN169arU0Hx6ZpaZLWD3r9//7y00dHRQuyge3p6\nOHLkCHv37qWrq4uDBw+yb98+KpUKg4ODbNu2je3bt887G69mBx1n06ZNzMzMUKlUWL9+/f+luR20\n4zhN09MDu3YVM/VTDbeDbmwHbWZX2wAYHh5mfHycVatWzSvrdtCO41y3tKIdtJnR19fHxYsXWbJk\nCUuXLuXQoUN0dXUBbgftdtBVcF3puN50uR304sLtoB3HcZxEuB204zhOC+N20I7jNE10le442cij\nH/kA4Dgl0tHRweTkpA8CTibMjMnJSTo6OjLV41NAjlMi3d3djI2NMTExUUj9ly5dynxQKALXlY4k\nujo6Ouju7s7Ujg8AjlMibW1trFy5srD6h4aGWLt2bWH1N4vrSkdZunwKyHEcp0XJNABIWibpBUln\nwndXlTy3SvqdpD9Jek3S17O06TiO4+RD1iuAncAxM1sNHAvrc/k38A0zWwPcDXxF0pqM7TqO4zgZ\nyToAfBZ4Jiw/A3xubgYzO29mJ8PyP4BR4JaM7TqO4zgZyXoT+ANmdj4svw3M90qNIemDwFrglTp5\n+oC+sPqOpNeb1LYc+HuTZYvEdaXDdaXDdaXjetR1e+MsEQ0HAElHgZuqJH07vmJmJqnmw82S3gP8\nAnjUzGZq5TOzJ4EnG+lqhKRhM7uzcc5ycV3pcF3pcF3paHVdDQcAM7u3VpqkC5JWmNl5SSuA8Rr5\n2ogO/gfMbKBptY7jOE5uZL0H8BvgobD8EPDruRkU+bT+FBg1s+9nbM9xHMfJiawDwF7gU5LOAPeG\ndSTdLOlwyPMx4EFgk6RT4bMlY7tJyDyNVBCuKx2uKx2uKx0trUvuSeI4jtOa+JvAjuM4LYoPAI7j\nOC3KNT0ASNoW7CX+I6nmI1OS7pP0uqQ3Je2MbW9oZdGkriQWGXfE7omckjQj6dGQ9rikc3nfM0n6\n90p6S1IltD2ctnwRuupZiuQZr1p9JZYuST8I6a9KWpe0bBYS6PpC0FOR9JKkj8bSqu7PErVtlDQd\n2z+7k5YtWNe3YppOS7oiaVlIKyRmkp6WNC7pdI30cvtX0t+OXIwf4EPAHcAQcGeNPO8CzgKrgHZg\nBFgT0r4L7AzLO4Hv5KQrVb1B49vA7WH9ceCbBcQrkS7gLWB51r8rT13ACmBdWL4ReCO2H3OJV72+\nEsuzBXgeEJG1yStJyxasawPQFZY3z+qqtz9L1LYReK6ZskXqmpP/fmCw6JgBHwfWAadrpJfav67p\nKwAzGzWzRm8K3wW8aWZ/MbPLwLNEFhaQwMqiSdLW2wucNbO/5tR+LbL+vQsWLyvHUqReX4lr3W8R\nLwPvVfQOTJKyhekys5fMbCqsvgxkM4rPUVtBZfOu+/PAz3NquyZm9iJwsU6WUvvXNT0AJOQW4G+x\n9TH+d+BIZWWRgrT1PsD8zve1cAn4dF5TLSl0GXBU0h8VWXOkLV+ULqCmpUge8arXVxrlSVK2WdLW\n/TDRWeQstfZnmdo2hP3zvKQPpyxbpC4kvRu4j+hl1VmKjFk9Su1fi/4HYVTHisLM5r141ixm9a0s\n0uhKU6+kduAzwK7Y5h8DTxB1wieA7wFfLlHXPWZ2TtL7gRck/TmcuSQtX5SuWpYiTcfrekPSJ4kG\ngHtimxvuz4I5CdxmZu+E+zO/AlaX2H4j7gd+b2bxM/OFjlkpLPoBwOpYUSTkHHBrbL07bANIZGWR\nVpcSWmQENgMnzexCrO6ry5J+AjxXpi4zOxe+xyX9kujy80UWOF6qYSmSJV5zqNdXGuVpS1C2WZLo\nQtJHgKeAzWY2Obu9zv4sRVtsoMbMDkv6kaTlScoWqSvGvCvwgmNWj1L7VytMAf0BWC1pZTjbfoDI\nwgISWFk0SZp65809hoPgLFuBqk8MFKFL0g2SbpxdBj4da3/B4iXVthTJMV71+kpc6xfD0xp3A9Nh\n+ipJ2WZpWLek24AB4EEzeyO2vd7+LEvbTWH/IekuouPOZJKyReoKejqBTxDrcyXErB7l9q+873KX\n+SH6Zx8D/glcAH4btt8MHI7l20L01MhZoqmj2e3vI/ohmzPAUWBZTrqq1ltF1w1E/widc8r/DKgA\nr4advKIsXURPGYyEz2uLJV5EUxoWYnIqfLbkHa9qfQV4BHgkLAv4YUivEHv6rFY/yylGjXQ9BUzF\nYjPcaH+WqO2roe0RohvUGxZDzML6l4Bn55QrLGZEJ3vngX8RHbseXsj+5VYQjuM4LUorTAE5juM4\nVfABwHEcp0XxAcBxHKdF8QHAcRynRfEBwHEcp0XxAcBxHKdF8QHAcRynRfkvl6jwNzwouHMAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x17d30dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_array = sess.run(x_vals)\n",
    "plt.plot(x_array, l2_y_out, 'b-', label='L2 Loss')\n",
    "plt.plot(x_array, l1_y_out, 'r--', label='L1 Loss')\n",
    "plt.plot(x_array, phuber1_y_out, 'k-.', label='P-Huber Loss (0.25)')\n",
    "plt.plot(x_array, phuber2_y_out, 'g:', label='P-Huber Loss (5.0)')\n",
    "plt.plot(x_array, phuber3_y_out, 'b.', label='P-Huber Loss (3.0)')\n",
    "plt.ylim(-0.2, 0.4)\n",
    "plt.legend(loc='lower right', prop={'size': 11})\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Categorical Predictions\n",
    "\n",
    "-------------------------------\n",
    "\n",
    "We now consider categorical loss functions.  Here, the predictions will be around the target of 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Various predicted X values\n",
    "x_vals = tf.linspace(-3., 5., 500)\n",
    "\n",
    "# Target of 1.0\n",
    "target = tf.constant(1.)\n",
    "targets = tf.fill([500,], 1.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hinge Loss ----------> Classification\n",
    "\n",
    "The hinge loss is useful for categorical predictions.  Here is is the `max(0, 1-(pred*actual))`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Hinge loss\n",
    "# Use for predicting binary (-1, 1) classes\n",
    "# L = max(0, 1 - (pred * actual))\n",
    "hinge_y_vals = tf.maximum(0., 1. - tf.multiply(target, x_vals))\n",
    "hinge_y_out = sess.run(hinge_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cross Entropy Loss ----------> Classification\n",
    "\n",
    "The cross entropy loss is a very popular way to measure the loss between categorical targets and output model logits.  You can read about the details more here: https://en.wikipedia.org/wiki/Cross_entropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Cross entropy loss\n",
    "# L = -actual * (log(pred)) - (1-actual)(log(1-pred))\n",
    "xentropy_y_vals = - tf.multiply(target, tf.log(x_vals)) - tf.multiply((1. - target), tf.log(1. - x_vals))\n",
    "xentropy_y_out = sess.run(xentropy_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid Entropy Loss ----------> Classification\n",
    "\n",
    "TensorFlow also has a sigmoid-entropy loss function.  This is very similar to the above cross-entropy function except that we take the sigmoid of the predictions in the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# L = -actual * (log(sigmoid(pred))) - (1-actual)(log(1-sigmoid(pred)))\n",
    "# or\n",
    "# L = max(actual, 0) - actual * pred + log(1 + exp(-abs(actual)))\n",
    "x_val_input = tf.expand_dims(x_vals, 1)\n",
    "target_input = tf.expand_dims(targets, 1)\n",
    "xentropy_sigmoid_y_vals = tf.nn.sigmoid_cross_entropy_with_logits(logits=x_val_input, labels=target_input)\n",
    "xentropy_sigmoid_y_out = sess.run(xentropy_sigmoid_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Weighted (Softmax) Cross Entropy Loss ----------> Classification\n",
    "\n",
    "Tensorflow also has a similar function to the `sigmoid cross entropy` loss function above, but we take the softmax of the actuals and weight the predicted output instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Weighted (softmax) cross entropy loss\n",
    "# L = -actual * (log(pred)) * weights - (1-actual)(log(1-pred))\n",
    "# or\n",
    "# L = (1 - pred) * actual + (1 + (weights - 1) * pred) * log(1 + exp(-actual))\n",
    "weight = tf.constant(0.5)\n",
    "xentropy_weighted_y_vals = tf.nn.weighted_cross_entropy_with_logits(logits=x_vals,\n",
    "                                                                    targets=targets,\n",
    "                                                                    pos_weight=weight)\n",
    "xentropy_weighted_y_out = sess.run(xentropy_weighted_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Categorical Losses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd8zdf/wPHXyZZBCEJsESKyNIlVRawa39q7jS1qVGu1NjWKDmp2UMTeVVVaK6FGrYq9olaomokE2ef3xyfuz0jIuLk34zwfj/vI/dx77ue8XXze+Xw+55y3kFKiKIqi5D0mxg5AURRFMQ6VABRFUfIolQAURVHyKJUAFEVR8iiVABRFUfIolQAURVHyKL0kACGElRDisBDihBDijBDi8xTaWAoh1gghwoQQh4QQZfXRt6IoipIx+joDiAXqSym9AG+giRCixkttegEPpZQVgJnAdD31rSiKomSAXhKA1EQnb5onP16eYdYSCEp+vh5oIIQQ+uhfURRFST8zfe1ICGEKHAMqAPOklIdealICuAEgpUwQQkQCDsC9l/YTCAQC5MuXz6dUqVIZiicmJobw8HDMzc0pVaoUJibGvd2RkCC4ds0GExNJmTJPMDHJXjOwk5KSjP4dpSQpKYn8V66QYGtLjKOjscPRyc7fl4or7XJjXBcvXrwnpSySpsZSSr0+AHsgGHB/6fXTQMnnti8DhV+3Lx8fH5lRwcHBcuvWrdLExES2adNGJiYmZnhf+rJ3r5SmpomyZUsps0E4LwgODjZ2CCkKDg6Wslw5KT/4wNihvCBbf1/ZkIorfTITF3BUpvF4rffUJ6WMSE4ATV566yZQCkAIYQYUAO7ru//nNW3alG+++YaNGzcybty4rOwqTd55Bz788DK//ALTphk7mhxkyBBo3drYUShKrqOXS0BCiCJAvJQyQgiRD2jEqzd5NwPdgINAO2B3crbKUh9//DFnzpxhypQpuLm50aVLl6zu8rXatr3Jw4cujBkDvr7QuLFRw8kZBg40dgSKkivp6wygOBAshDgJHAF2SCm3CCEmCiFaJLf5CXAQQoQBQ4AReur7tYQQzJs3j7p169KzZ08OHz5siG5fEw/8+CO4u0PnznD1qlHDyRkePIBbt4wdhaLkOno5A5BSngSqpvD6uOeexwDt9dFfellYWLBhwwaGDx9OhQoVjBHCC2xs4OeftTOANm1g/37Il8/YUWVj3bpBeDgcP27sSBQlV8l+t7+ziIODA4sWLaJQoULExsby+PFjo8bj7AzLl2vHtH79QJVleA0bGzDy35ei5EZ5JgE8k5iYSOPGjQkICMAAtyBeq3lzmDABgoLg+++NGkr2ZmMDT54YOwpFyXX0Ng8gpzA1NaVjx44UKVKE7DAPbexYOHoUPv4YvLygVi1jR5QNqTMARckSeS4BAPTv31/3/MGDBxQqVMhosZiYwLJl2v2Adu3g77+hWDGjhZM9qQSgKFkiz10Cet5vv/1GmTJlOHjwoFHjsLeHjRshMhI6dID4eKOGk/289x7MmKFulCiKnuXpBFCjRg2KFStGq1atuHbtmlFj8fSEhQvhzz9h+HCjhpL91KqlzQXIBpfsFCU3ydMJwMHBgV9//ZXY2FhatGhBdHT0mz+UhTp3hk8+gVmzYOVKo4aSvURFwcmTEBNj7EgUJVfJ0wkAwNXVlbVr13LmzBnef/99kpKSjBrPl19CnTrQu7d2zFOAP/7Q7pBfvGjsSBQlV8nzCQCgcePGfPvtt2zevJkRIwwyQTlV5uawZg0ULKhNEnv40KjhZA/582s/Hz0ybhyKksuoBJBswIAB9O/fn6+++ooFCxYYNZZixWD9erh+HT74AIx8UmJ8KgEoSpZQCSCZEIJZs2bRpEkT+vfvz86dO40aT82a2r2ArVth4kSjhmJ8BQpoP1UCUBS9UgngOWZmZqxZswY/Pz/is8FYzA8/hO7d4fPPYcsWY0djROoMQFGyRJ6cCPY6+fPnZ//+/bpZwomJiZiamholFiFg/nztZvAHH2gzhrPBWnaG5+AAP/0Eb79t7EgUJVdRZwApeHbw/+6776hbty5Pnz41Wiz58sGGDWBqqt0UzpMTYq2soGdPqFTJ2JEoSq6iEsBrFC1alKJFixp9aGjZsrBqFZw+DX365NEJsceOwYULxo5CUXKVTCcAIUQpIUSwEOKsEOKMEOLjFNrUE0JECiFCkx/Gr8+YBm3btmXDhg3Y2NgY/Z5A48YwZYqWCGbPNmooxtGqFUx/ucicoiiZoY8zgARgqJTSDagBDBBCuKXQ7k8ppXfyI8eMaxFCcPfuXapVq0ZQUJBRYxkxQjsODh0Ke/YYNRTDs7dXkyIURc8ynQCklP9KKf9Ofh4FnANKZHa/2Ym9vT0ODg706dOHkJAQo8UhhFY7wNlZWzTu5k2jhWJ4Dg5aaUhFUfRGr/cAhBBl0UpDHkrh7ZpCiBNCiG1CiCr67DermZubs379eipUqECbNm24YMRr0fnza+UknzzRlo+OjTVaKIZVqBDcv2/sKBQlVxH6qoolhLAF9gBTpJQbX3ovP5AkpYwWQjQDZkkpXVLZTyAQCODo6OizevXqDMUTHR2Nra1thj6bmn///Zf+/ftjbW3N/PnzKfBsgpIR4tqzpwgTJlShRYubDB58KdP7y4rvSx+exVXx669xOHiQgxs2GDskIPt/X9mNiit9MhOXv7//MSmlb5oaSykz/QDMgT+AIWlsfxUo/KZ2Pj4+MqOCg4Mz/NnXOXDggLS0tJS1a9eWMTEx6f68PuP69FMpQcrFizO/r6z6vjJLF9fx41Ju327UWJ6X7b+vbEbFlT6ZiQs4KtN47NbHKCAB/ASck1LOSKVNseR2CCGqoV16ypHn8zVr1iQoKIh9+/bRu3dvo9YVnjIF6tfXZgz//bfRwjAMb29o1MjYUShKrqKPmcBvAwHAKSFEaPJro4DSAFLK74F2QD8hRALwFOgkjXnkzKSOHTsSFhbGmDFjcHZ2ZsKECUaJw8wMVq8GHx9tktixY9q90lzp9m346y8t4z1bGkJRlEzJdAKQUu4DXluqSUo5F5ib2b6yk1GjRhEWFkZYWBhJSUmYmBhnTl2RItpM4dq1tYIy27Zps4Zznb/+gtattSz31lvGjkZRcgU1EziDhBD8+OOPLFu2DBMTE6NeCvLz09YM2rEDxuWIKXYZ8OzURo0EUhS9UQkgE8zNzRFCcOXKFapVq8aJEyeMFkuvXhAYCF98oQ0TzXVUAlAUvVMJQA/Mzc2Jj4/nsZFXaps9G6pVg27d4Px5o4aif4UKaT/VZDBF0RuVAPSgZMmS/P3339SqVQuAhIQEo8RhaalVErOy0m4KR0UZJYys8SwBqDMARdEblQD05NlN4KlTp9KkSRPi4uKMEkepUlpN4QsXoEePXLRyqIUFBAdrfyhFUfRCJQA9K1myJLt27TLqHAF/f/jyS2100NdfGyWErFGvHpQsaewoFCXXUBXB9CwgIIBr164xduxYypQpw6RJk4wSx5AhcPiwtoLoW29BgwZGCUO/duyAuDho3tzYkShKrqASQBYYPXo0165dY/LkyZQuXZo+ffoYPAYhtCqKp09Dp07a8PnSpQ0ehn599ZVWF1glAEXRC3UJKAsIIfjuu+9o0qQJ/fr1Y9u2bUaJw9ZWGxIaFwdt20JMjFHC0B8HB7h3z9hRKEquoRJAFjEzM2Pt2rV4enrSvn17/jbSYj0VK8LSpVpB+YEDjRKC/jg6wn//GTsKRck1VALIQnZ2dvz2228ULlyY5s2bc/XqVaPE0bIljBmjXRJasMAoIehH8eIQHa09FEXJNJUAsljx4sXZunUrBQsWJCIiwmhxTJgA776rnQUcPmy0MDLHyUn7+e+/xo1DUXIJlQAMwM3NjVOnTuHt7Q1glALzpqawciWUKKHdD7hzx+AhZN7//qdNcS5b1tiRKEquoBKAgZgmL9G5YMECWrZsaZQkUKiQNjfg3j3o2BGMNGE54woWhEqVwNzc2JEoSq6gEoCBFS9enPLlyxtt+eiqVeHHHyEkBEaONEoIGZeQADNnwt69xo5EUXIFlQAM7H//+x9z587F1NSUyMhIo8QQEAADBmizhNeuNUoIGWNqCqNGwa+/GjsSRckV9FESspQQIlgIcVYIcUYI8XEKbYQQYrYQIkwIcVIIkecreoSHh+Pu7s6MGSlW0cxyM2ZArVrQsyecOWOUENJPCG0kkLoJrCh6oY8zgARgqJTSDagBDBBCuL3UpingkvwIBL7TQ785WvHixalZsyZDhw5lxYoVBu/fwgLWrQM7O63QVnR0DikjVrw43Lpl7CgUJVfIdAKQUv4rpfw7+XkUcA4o8VKzlsDS5KL1fwH2Qojime07JzM1NWXZsmX4+/vTvXt3/vjjD4PH4OSkJYErV2Dq1MokJRk8hPRzclJnAIqiJ0KfK1YKIcoCewF3KeWj517fAkxLrh+MEGIX8JmU8mgK+whEO0vA0dHRZ/Xq1RmKJTo6Gltb2wx9Niu9HFd0dDSDBw8mPDycGTNmULlyZYPHtHFjCebMcaFXr3/44IPrBu//dV7+virMnk2x7dvZt2WLEaPKOf++sgsVV/pkJi5/f/9jUkrfNDWWUurlAdgCx4A2Kby3Baj93PYuwPdN+/Tx8ZEZFRwcnOHPZqWU4vr3339luXLlZOHCheWFCxcMHlNSkpQNG96WQki5bZvBu3+tV76vhw+ljI42SizPy0n/vrIDFVf6ZCYu4KhM43FbL6OAhBDmwAZghZRyYwpNbgKlntsumfyaAhQrVozt27cjhKBx48bcMvA1biFg6NALeHpCly7aJaFsy94ebGyMHYWi5Ar6GAUkgJ+Ac1LK1Ia0bAa6Jo8GqgFESinVhdznVKhQgW3btnH//n0aNWrEo0eP3vwhPbKySmLDBq2CWJs28OSJQbtPu1u3YOhQCA01diSKkuPp4wzgbSAAqC+ECE1+NBNCfCiE+DC5zVbgHyAMWAD010O/uY6Pjw+//vorrVu3xs7OzuD9Oztry0WcOAEffphNy0kmJGhjWI8cMXYkipLjZbogjNRu7Io3tJHAgMz2lRfUq1ePevXqAfDPP/9QvHhx8uXLZ7D+mzbVFo4bPx6qV9cmjGUrTk5gZgZGWllVUXITNRM4m4qKiuLtt9+mX79+Bu97zBht3bVPPoH9+w3e/euZmWl1ga9dM3YkipLjqZKQ2ZSdnR3Tp0+nevXqBu/bxASWLQM/P2jfXisnWTw7zdooW1adASiKHqgzgGysa9euVKpUCSklmzZtejaE1iDs7bVykpGR0KEDGGHx0tSVLavVBlYUJVNUAsgBNm/eTOvWrRlp4OU73d21KmL79sGwYQbt+vUWLoSTJ40dhaLkeCoB5AAtWrSgX79+TJ8+nWnTphm0706dYPBgmD0bli83aNepM80h6xYpSjanEkAOIIRg7ty5dOnShZEjR/Ldd4ZdS2/6dKhbFwIDtSGiRnflinZdKsfWtlSU7EElgBzCxMSEJUuW0KJFCwYMGMCSJUsM1re5OaxZo1UUa90aHjwwWNepB7RuHfz9t5EDUZScTSWAHMTc3Jw1a9bQsGFDevbsycqVKw3Wt6MjrF8P4eHwwQcYd+VQJyewtoaLF40YhKLkfCoB5DBWVlZs2rSJevXq0bVrV9atW2ewvmvUgDlzYNs2+Pxzg3X7KhMTcHFRCUBRMkklgBzI2tqazZs3U6NGDT777DNiY2MN1ndgIPToARMnGrkyY8WKcOGCEQNQlJxPTQTLoWxtbdm6dSsPHjzA0tLSYP0KAfPmaaMwAwK0JXlcXAzW/f+rWlWbDZyUpJ0RKIqSbup/Tg6WP39+ypYtS1JSEoMHD2b79u0G6TdfPtiwQVuVoU0bePzYIN2+aORIOHRIHfwVJRPU/55c4PHjxwQHB7N3716D9VmmDKxaBWfPQu/e2XTlUEVRXkslgFzAzs6Offv2MWnSJAASEhIM0m+jRjBlCqxeDd9+a5Au/19MDNSqBQaeE6EouYlKALmEra0tQgjOnj1L5cqV2bdvn0H6/ewzbW7A8OEQEmKQLjVWVvDPP6ougKJkgr5KQi4SQtwRQpxO5f16QojI5wrGjNNHv8qr7O3tMTU1pUmTJuzZsyfL+xMClizRbgR37KjNEzAYd3c4dcqAHSpK7qKvM4AlQJM3tPlTSumd/Jiop35TFPYgjE9CP+HCvbw3TNDJyYng4GBKly5N06ZN2b17d5b3mT8/bNyolZFs1w4MNirVwwPOnDHyrDRFybn0kgCklHsBYy8QoHM98joP4h6Q3zK/sUMxiuLFixMcHIyzszPNmzdnx44dWd5n5coQFKQNzPnkkyzvTuPhAU+fapeCFEVJN6GvNeaFEGWBLVJK9xTeqwdsAMKBW8AwKeWZVPYTCAQCODo6+qxevTpD8TyKekR+Oy0BfH3xa9zzu9Ok2JtOUrJedHQ0tra2BukrIiKCoUOHcuPGDSZPnky1atWyPK4ffyzPqlWl+fTT8zRtejvT+3tdXDaXL1Nu0SL+CQzkSZkyme5LX3EZk4orfXJjXP7+/seklL5paiyl1MsDKAucTuW9/IBt8vNmwKW07NPHx0dmVHBwsJRSysdxj2WdxXXklL1TMrwvfXoWl6Hcu3dPent7SwsLC7lly5ZU2+krrvh4KRs2lNLSUsqjRzO/P0N/X2ml4kofFVf6ZCYu4KhM43HbIKOApJSPpJTRyc+3AuZCiMKG6Nva3JqQbiF8+vanAOy9tpc+m/sQGRNpiO6NzsHBgV27duHh4UHr1q05f/58lvZnZqbND3B01CaJ3buXpd1pDLgUhqLkJgZJAEKIYkIIkfy8WnK/9w3Rd3KfmJloq16E3g4l5FoI5qbmhure6AoVKsTOnTuZO3culSpVyvL+ChfWZgr/9x907gyJiVnYWd++2mggRVHSTV/DQFcBB4FKQohwIUQvIcSHQogPk5u0A04LIU4As4FOyacqBjeo+iBO9TuFtbk1iUmJ9PylJ4dv5v7CIvb29gQGBiKEIDQ0lKCgoCztz9cX5s+HnTthzJgs7KhcOQgLywZFChQl59HLYnBSys5veH8uMFcffemDlZkVAFcjrrL98nYalW9EtRKp3yDNbb755hv27t1Lu3btsLGxybJ+evbUinZNmwZ+ftolIb3z89N+Hj0KjRtnQQeKknvl6dVAnQs5c2HgBazNrQFYe2Ytdx7foZ9vP0xNcm/d2YULF3L79m1sbGxISkoi+epclpg1C0JDoVs3cHMDV1c9d+Djo/1UCUBR0i3PLwVhY2GjOwBuvrCZFadWZOkBMTuwtLSkTJkySCkZMGAAQ4cOJauuyFlaapXErK21JSOiovTcgb29Ng1ZLQmhKOmW5xPA85a1Xsa297dhIkyIjotm0LZB3Iq6ZeywsoyUEgsLC2bOnMnXX39NYhbdrS1ZUqspfOkSdO+eBSuHfvqptg6FoijpohLAc4QQ2FvZA3DgxgEW/L2A65HXjRxV1jExMeHbb79lzJgxbN26lc6dO2dZdbF69eDLL7UlI778Us87790bOnXS804VJffL0/cAXqexc2Ouf3KdIjZFAJh3eB4l85ekpWtLI0emX0IIJk2axP379/nuu++4f/8+P//8M/nz638ZjcGDtZvCo0Zpl+4bNtTTjpOStPrAtrba6YaiKGmizgBe49nBPzEpkSUnlrD6TMaWpcgJOnTowNKlS9m7dy916tTh33//1XsfQsDChdq6QZ06aRUd9SImBqpUgR9/1NMOFSVvUAkgDUxNTDnY6yDfNdeKj9yIvMHg3wfz8OlDI0emXwEBAWzZsoWwsDBq1qzJhSwoum5rCz//DPHx0LatduzONGtr8PICA9VAUJTcQiWANDIzMdPdH9h9ZTcL/l5AZGzuW07i3XffJSQkhCdPntCzZ88sGR3k4gLLl8OxY9C/v55uCtetCwcPQlycHnamKHmDSgAZ0M27G9c+uUZZ+7IAfB7yOSFXQ4wakz75+vpy8OBBli9fjhAiS5LAe+/B2LGweLGertzUqaOdTqjhoIqSZioBZJCDtQMAUbFRLDmxhO2Xtxs5Iv1ydnamXLlyJCUl0aNHDxYuXKj3PsaPh6ZN4aOP4K+/Mrmzd97Rfu7dm+m4FCWvUKOAMsnO0o6z/c8i0X5LPnrrKJvOb2Jk7ZHYWGTdMguGEhsby3///cfdu3f1vm9TU+1SkK+vVkns2DFtFdEMKVwYfv/9/2cGK4ryRuoMQA/ymefTLSfxR9gf/HT8J5Jk7ihTmC9fPn799VdGjBgBQGhoKI8fP9bb/gsV0m4KP3igzeVKSMjEzt59V0sEiqKkiUoAeja6zmjO9j+LnaUdUkp6b+6d4+8PmJmZIYQgKiqKhg0bUrduXW7d0t8MaS8v7T7Anj3w2WeZ2NH9+/D115DFNQ8UJbdQCSALFMxXEIBbUbfYfWU3YQ/CjByRftjZ2bF48WLOnz9P9erVOXHihN72/cEH2r2AGTMgg1VAtbGlw4fDL7/oLS5Fyc1UAshCJfKX4NyAc/Tw7gHAhrMbmHlxJtFx0UaOLOPee+899u3bh5SS2rVrs2XLFr3t++uv4e23oVcvOH06AzsoVgw8PeGPP/QWk6LkZvoqCLNICHFHCJHif1uhmS2ECBNCnBRCvKWPfnMCSzNL3dLSF+9f5FzUOfKZ5TNyVJnj7e3N4cOHqVixIi1atGDq1Kl6GSpqYQHr1kH+/NrKoRERGdhJkybahDC9LzuqKLmPvs4AlgBNXvN+U8Al+REIfKenfnOUke+MZH7V+ZiamBKTEIN/kD+/h/1u7LAyxMnJiT///JNOnToxatQoOnToQHR05s9sihfXlo++ehW6dtWW+UmXJk20S0G7d2c6FkXJ7fSSAKSUe4HX1eRrCSxNLlr/F2AvhCiuj75zmme1iW9H3yYiJkK3nRNHDVlbW7NixQq++uorNm7cSJ06dYiPj8/0ft9+G2bOhF9/heXLy6T/wwULamUiFUV5LaGvWZ5CiLLAFinlKxW6hRBbgGlSyn3J27uAz6SUR1NoG4h2loCjo6PP6gzeEYyOjsbW1jZDn81Kz8eVJJMwEVoOXnV9FSciT/C52+dYmloaNa6MOHr0KP/++y/vvfeeXuKREqZOdWXnTkemTj1F9eppr/kr4uKQFhZ6iSM1OeHfV3ai4kqfzMTl7+9/TErpm6bGUkq9PICywOlU3tsC1H5uexfg+6Z9+vj4yIwKDg7O8GezUmpxzT88X36w8QPd9uO4xwaKSKPP72vz5s3yyy+/lElJSZnaz+PHUjo7R0l7eynDwjKwg0z2/zo57d+Xsam40iczcQFHZRqP24YaBXQTKPXcdsnk15Rk/fz6saz1MgDuPr5LmW/LEBQaZOSoMmbTpk2sWbMm05eDrK1h4sTTCKGtHPrkSRo/GBurLQ73zTeZ6l9RcjtDJYDNQNfk0UA1gEgppf4XnM8lJJJWlVpRvWR1AB4+fcjjOP3Nvs1qCxcuZMeOHVhYWPDw4UOOHTuW4X05OcWwciWcPAmBgWlcOdTSEp4+1e4mK4qSKn0NA10FHAQqCSHChRC9hBAfCiE+TG6yFfgHCAMWAP310W9uVdSmKAtaLMC1sCsAo3ePpvK8yjyNf2rkyNJGCEHBgtpkuFGjRlGzZk3mzp2b4aGiTZrAxImwYgXMnZvGD7VpA4cOQXh4hvpUlLxAX6OAOkspi0spzaWUJaWUP0kpv5dSfp/8vpRSDpBSOkspPWQKN3+V1AV4BvDp25+Sz1ybP3DgxgESk7KmgLu+TZkyhXfffZePPvqIjh078ujRowztZ9QoaNEChgxJY92XNm20n5s2Zag/RckL1EzgHKBmqZoMrDYQgEv3L/HO4nf46sBXRo4qbQoVKsQvv/zC9OnT2bhxIz4+Phw+fDjd+zExgaVLoVw5aN8e3lixsmJFrUzkxo0ZC1xR8gCVAHIY50LOrG67ml5VewFw6r9T/Hrh1ywp2qIvJiYmfPrpp4SEhBAXF0etWrWYMmUKiYnpO4spUEA7nkdFaUngjcW/hg3T7h4ripIilQByGBNhQvsq7XUF62cfmk23Td2Iisv+Sx/Url2bEydO0KFDB8aMGUO9evW4evVquvbh7g6LFsH+/TB06Bsad+8OAwZkNFxFyfVUAsjh5jefT0j3EPJb5kdKyZA/hrD/+n5jh5Uqe3t7Vq5cyfLlyzl58iTffZf+VUE6dNAO/nPnwrJlb2gcEaEuAylKKlQCyOHMTc3xdPQE4L/H/7H69Gr+/vdvgOcn4mU777//PidPnuTzzz8H4OTJk+mqOjZtGtSrpw0NPX78NQ0XLtQuA126lLmAFSUXUgkgFylmW4zLgy7T17cvAL9c+IV3Fr/DrSj9FW/RpzJlymBlZUVSUhKdOnWiZcuWaU5YZmawZg04OGjH9weprRTRuTMIoY0hVRTlBSoB5DL5zPNhYaqtg5OYlIiFqQVFbYoCcPPRzWx5RmBiYsLatWv59ttvEULw5MkT7ty588bPFS0KGzbAzZvQpQukeE+5RAnw99eKD6d7aVFFyd1UAsjF2rq1ZXe33ZiZmJGYlEi9oHp03dTV2GGlyN3dnWrVqgEwfvx43NzcWL169RsTVvXqMGeOVgNmwoRUGvXoAZcvQ3CwfoNWlBxOJYA8ZGydsXT11BJATEIMq06tIj4x88s361uPHj1wdnamc+fOjBgxgsuXL7+2fZ8+WhWxyZNh8+YUGrRrp1WfDwnJkngVJadSCSCPMDUxpatXVxo5NwJg3Zl1dNnYhYPhB40c2avc3Nw4cOAAs2bN4vTp07i7uzNlyhRiY2NTbC+ENiLI1xcCAuDixZcaWFlpheInTcr64BUlB1EJII963/N9grsF807pdwBYH76e0btGZ5t7BKampgwaNIglS5bw3nvvMWbMGLy9vQlJ5bd4KyvtfoCFhbYKxCvFyYpo8yZISMjSuBUlJ1EJII8yESbUK1sPIQQA159c5/Td07rtB0/TXoAlKxUpUoS1a9fy22+/ERsbi7+/P++//z4JKRzIS5eG1avh3DntktAruWzqVK1ovLoZrCiASgBKsiEVh7ChwwYA7j25R+mZpZnfqn6KAAAgAElEQVR7OK1Lb2a9Zs2acfr0acaMGYOFhQVmZlopzZcTQYMG2nF+7VqYMeOlnZQtq2WHHTsME7SiZHMqASg6z+oTmwpTBtcYTINyDQC4/OAyi44vIiYhxpjhYW1tzaRJk1i0aBEAZ86coUKFChw6dOiFdsOHa3MDPvvspYE/bdpAsWJawWFFUVQCUF5VMF9BJtWfROUilQFYfXo1H275kIiYCMD4BeyfXaaKj4/HxcUFZ2dnAN1S00LA4sXg4gIdO8KNG8kftLSEQYO0MaMnTxojdEXJVvRVEKaJEOKCECJMCDEihfe7CyHuCiFCkx+99dGvYhij3hnFiQ9PUMy2GADvb3yfflv6GTkq8Pb2ZseOHRQuXBgpJfXr16dx48YcPXoUOzv4+WeIidFGgeoGEPXtCzY2KVwfUpS8J9MJQAhhCswDmgJuQGchhFsKTddIKb2THwsz269iOEII3dmAlJLS+UtTMn9J3furTq0iMibSWOEBkJiYSJcuXfj777/x8/OjXbt2wHmCguDwYe0Xf0CbD7B0KYwfb8xwFSVb0McZQDUgTEr5j5QyDlgNtNTDfpVsSAjB9EbTGV1nNACn75ymy8YuLD2xFDDeAnRmZmYMGTKEf/75h/Hjx/PHH39QpUoVNm4MoE+fs/z4I/z0U3LjNm20yjKKkseJzP5nFUK0A5pIKXsnbwcA1aWUA59r0x2YCtwFLgKDpZQ3UtgdQohAIBDA0dHRZ/Xq1RmKKzo6Gltb2wx9NivltriklFyMvkiJfCWwNbNl3719LLm6hCnuU3C0cjRaXBEREaxatYrNmzcTGxuLvf17REWNZ84ciatrFNZXrlD+p584P3w4CQUKGCyurKbiSp/cGJe/v/8xKaVvmho/+40tow+gHbDwue0AYO5LbRwAy+TnfYHdadm3j4+PzKjg4OAMfzYr5fa4tl7cKpsubyrjE+N12zsu7zBaXHfv3pWjR4+Wdnb5palpcVmyZJy8c0dKeeqUlEJIOXq0UeLKKiqu9MmNcQFHZRqP3/q4BHQTKPXcdsnk155PMvellM9uwy0EfPTQr5INNXVpytb3t+qGlH6x7wvG7B6je//ek3sGjadw4cJMnjyZ69ev8dNPG7l715yOHeNpNWYsIXXrwqxZcP++QWNSlOxCHwngCOAihCgnhLAAOgEvLMklhCj+3GYL4Jwe+lVygB0BO1jZdiUAsQmxVJxTkdG7Rhs8Dnt7e7p1q8H330Nw8DX27DlNVLt28PgxDydPJiIiwuAxKYqxZToBSCkTgIHAH2gH9rVSyjNCiIlCiBbJzQYJIc4IIU4Ag4Dume1XyRmszKwoX7A8AIkykbF1xtLMpRmg1SdotqIZobdDDRZP9+7w4YcViIi4QEzRftChA1/Nm0epkiXp168fx19bXkxRchczfexESrkV2PrSa+Oeez4SGKmPvpScy9rcmsE1B+u2r0Rc4dy9c1iZWQFw6r9TnPjvBO3c2uleywrffguhoSb07AlvrRpPRwcHbkVGsmTJEr7//nv8/PwIDAykU6dO2fIGoaLoi5oJrBhN7dK1+WfQP7gWdgVg5amV9Pm1j65GwaX7l3gU/0jv/Vpawvr1YG0NzYdVptzUeSxZvpxbt24xe/Zsnjx5Qp8+fXBycqJv377s378/26ySqij6pBKAYlTPlnUAmNJgCqF9Q7GztAPg498/ZuBx3WhivU42K1FCWzAuLAy6dYOkTZspOHkyH330EadOnWL//v20bt2a5cuXU7t2bT777DO99a0o2YVKAEq2YSJMqFS4km57ov9E+jv3B7Thym/9+Bb9f+uvez+zv5XXrQtffw2bNsG+eSe05SH270cIQa1atQgKCuL27dssXbqULl26AHDkyBH8/Py4cuVKpvpWlOxAJQAl2/J18qWGQw1Au4E80G+g7gZydFw0TjOcWHZiWab6+Phj6NQJmu8aQoyDk/bCc9Xl7ezsCAgIwNvbW+s3OhozMzOKJBeYWbNmDVOnTuXChQuZikNRjEElACVHMDMxY3DNwfyv4v8AiIqNommFproRRmfunKHK/CocCj/0ut28QghYuBDKVrFhYMw3cOwY/PBDqu39/f05ePCg7uZwcHAwo0aNwtXVlSpVqjBmzBgOHjxI4nNJRFGyK5UAlBypuF1xFrVcxNul3wa0Ivcl7EpQ3E6bcvLzuZ+pvrA64Y/C37gvGxtt5dD1ph05ZNcAOWoU3L2bpji+//57bty4wZw5c3B0dGTq1KnUqlWLIkWK0KlTJ5YsWcLt27cz/gdVlCykEoCSK/g4+bA9YDulC5QGtDMGWwtbHG209Yim7ZtG9YXViUuMA9D9fKZCBVi+QtA1ah4LPWYjHQqnue+SJUsycOBAdu/ezb1791izZg2tWrVi79699OjRg4ED//9G9rZt23j48GFm/7iKohcqASi50nuV3mNX112Ym5oDUMKuBFWKVMHC1AKAnr/0pMbCGrr2VyOu0rBJDJ3HVyJwX1d++FGkUFT4zQoWLEiHDh1YtGgRN2/eJDQ0lLFjxwJw/fp1mjVrxrJl2n2Le/fusWnTJu6rpSgUI9HLRDBFye4CvAII8ArQbTd2bkzVYlV12x3WdcDO0o4d43Zx5Aismz2Yt5bsoNruw9qEgQwQQuDl5aXbLlasGHv27KFChQoA7NixQze6qEKFClSvXl338PLywtLSMkP9KkpaqQSg5Eldvbq+sD2u7jjMTMwwMYFlyyTFv17IolPRVBk8GpsfZtL/t/40d2lO84rNgYwNQbWwsKBOnTq67bZt27Jv3z727dvHoUOHCA4OZsWKFbq23t7e+Pr6Mm7cOBwdHZFSvjBvQlEyS10CUhTgfxX/R5MKTQAoWBB+aXGaUgc+IN+Ps4jc8QdbLm7h/L3zgDYEtcWBFiw6rhWnj0uM47eLv6V7pVMLCwvefvttPvvsMzZu3MjNmze5ceMG69ev5+OPP8bKyorly5frzgQmTZqEm5sbCQkJAJw5c4ZLly7pthUlvdQZgKK8RAhBkxpliPzyO670PkCB9v25fvMcSdb5AG3EUcOiDXEp5ALAxfsX+d+q/7GyzUo6e3Tmn4f/MHT7UMbWGctbxd8iOi6a29G3KWtfVrdMdmpKlixJyZIladu2LcALv/VXqlSJ+vXrY2am7WPIkCFs374dc3NzKlSogKurK/ny5ePatWu4urri4uJCwYIF1VmDkip1BqAoqejYy5bf2i6mYORV9o36DROh/XcpbF2Yj10+5p0y7wDgXNCZ/T3307B8QwDuP7nPxfsXSUzS5gL8ee1PXOa48Ff4XwAcu3WMPpv76Iao3ntyj3N3z+nWQHre8wfvjh07MnfuXN32tGnTWLx4MUOGDKFSpUqcO3eONWvW0L17d2rUqIGDgwM1avz/je45c+awfv163faTJ0/08j0pOZc6A1CU1+i3qg4Btc7zy0IX/uoNHh6vtslnno9apWrptv1K+HGm/xndtqejJ4tbLsa9qDugjTj69eKvjKurLZi78dxG+m7py/VPrlOqQCk2nN3AwuMLWdV2FfZW9pz67xQX7l+gZaWWmJuaE58Yj5mJGVWrVqVq1aovxLJz505Kly7NhQsXuHTpEtbP3cCeP38+fn5+tGvXDikljo6OmJiYUKJECZycnFL8Wa5cOYoWLarPr1TJRlQCUJTXMDeHGb+6EPIWTGx6kIXbSlDAo3S69lEifwm6e3fXbbd1a0tbt7a67cbOjVnRZoVuEltsYiz3n9zH1kKbbbzu7Dqm/DmFuDHa3IUJIROYdWgWj0Y+wkSYsPTEUg7cOMD3//seMzMz7uW7R5xzHEPeGwJoZySmJqacPXuW2FitMF9iYiJjx44lPDycW7ducfPmTUJCQrh169YL9xQCAwP54YcfSExMpHLlygwbNozAwEAiIiKYOnUqRYoUoUiRIhQuXJjChQtjb2+ve6hRTNmfXhKAEKIJMAswRasPPO2l9y2BpWilIO8DHaWUV/XRt6JktWLFYGNQFC6N/8fNupWxux2i1/2XtS9LWfuyuu0uHl3o4tFFtz2k5hA6VumIqYkpAPXK1sPSzFJ3SerKwyscvXVU137B3wvY9c8uXZL55I9P2Hd9H1c+voKVlRWDfx/M9UfX2fDpBgC+/etbomKjGFt3LElJSSw9spT79+/jKl0pVqwYp++cJjEuET8/PxwdHYmOi+bGrRt8++23xMW9OKHueVZWVnz11VcMHDiQO3fu0KRJE0aNGkWdOnW4evUqK1aswNbWFhsbm1R/Ojo6YmWVdbUh8rpMJwAhhCkwD2gEhANHhBCbpZRnn2vWC3gopawghOgETAc6ZrZvRTGUGo3s2N5jLo0Xd+HPhuNhYiOD9W1vZY+9lb1uu5FzIxo5/3//4+uNZ3y98brtbxp/Q0TM/5e47OHdgybOTXTbxWyLvbD/0Nuh3H+qTUYzMTFh8bnFAAztPhSAtxe9TT6zfOxcsRMAvwV+FLEuQkxMDFFRUTRe2ZiiZkXpW7wvkZGRzLw+k/zx+fGN9sXLy4vh24cTcS+Chw8fkpCQwPR903l87TGTxkzSAngLuAdcTw7IFXgA3IENGzbg6OvItVPXGPDBALZv3459eXsOBR9i5tSZWFpZYmpnio25DTYWNlhaWWKezxxrS2usrawZNGgQZcqU4fTp0+zevZsePXpgZ2fH6dOnOXv2LBcuXCA6Ohpzc3PMzMwwNzd/4VG5cmUsLCyIjIwkOjoaJycnhBDExcUhpcTc3BwTk5x7K1UfZwDVgDAp5T8AQojVQEvg+QTQEpiQ/Hw9MFcIIaSqsqHkII1+6syeP3fh/uf3jJjdgZMnjR3Rqy5dKsHJk4WAQs+9Wh+A2SHaVj4+oxww+6K2/RZLtPdna9vN5a8kyDjddvWEb1543zV2EBaRNsyZI4D8FHraDCuTolx+os2RQG7DyqISJUqM4fhxWPdoOPbRlen5/iFOn4apEQG4m7fgm29iiYt7zITYsngmtqTOk0HExkbzncO7uEa+i8ftlpw86cOkU87USOiFp2dXtmwtwkTK4RsVQGysE4+ePiascTAFjpTA+mBBYsVjHgRewWK3DeKQINa0Od/ZVMD73w78/cNKbkTU5jsa43ypDieX/wwF0ArU/gGcBwqi/Wq6A7gMgz7fyyaToRQ76cbhdUGM+OY0a59+iM1+B05t+wWKAk1A7DTF9I4ZFBMk1YvHfI81Zg8s+PDzP9gROwWbg4UJP3Gc9p/MJyRmBmK3KTfPHCfeMYanVR5S4GgJzGPyEef4hMcu92hpN4969fT2zyJVIrPHYCFEO6CJlLJ38nYAUF1KOfC5NqeT24Qnb19ObvPKwGkhRCAQCODo6OizevXqDMUVHR2dLcv5qbjSJ7vFFf8onvnjC7MptOqbGyuvIYHkEU757kOSOcTm17aLnIGYghDlpG2X3wkRZeCBC4hE8FgF/3lqD5N48FkA4TXg37fA7CnUmAX/NIBbfmARBe9MhQtNIdwN8knwnwCnmsGNsmD7LzSaAUdbwI3KUCAcms6BA63gugsUdIVmQ2FPBwhPgMK1ofkA2NkCbj6GYjeg2W/we0O4VRRKhEOzXfCrP9x2gNIdtfYb2sOdB1C+OTQbAKubw73rUPEmNDsBS6vDAytwuwVNz2O75nd+XZax4bv+/v7HpJS+aWmb7RLA83x9feXRo0df1yRVISEh1DNECk0nFVf6ZNe4tmzZR61atY0dxiv27dtH7doqrrTKrnHt37+P997LWFxCiDQnAH1cAroJlHpuu2Tyaym1CRdCmKGdeKkVsLKZ+Ph4wsPDiYmJMXYoOgUKFODcuXPGDuMVJUrY8N9/2S+uUqVUXOmRXeMqWdLmjf/uraysKFmyJObm5hnuRx8J4AjgIoQoh3ag7wR0eanNZqAbcBBoB+xW1/+zn/DwcOzs7Chbtmy2mT0aFRWFnZ2dscN4hYorfVRc6fOmuKSU3L9/n/DwcMqVK5fhfjJ9+1pKmQAMRLuNcg5YK6U8I4SYKIRokdzsJ8BBCBEGDAFGZLZfRf9iYmJwcHDINgd/RVFSJoTAwcEh02frepkHIKXcCmx96bVxzz2PAdrroy8la6mDv6LkDPr4v5pzB7AqiqIomaISgJJtlS1blrNnz77wmq+vLyEhIQCMGzeONWvWGCweIQTR0dEG609RsppaC0jJsSZOnGjsEBQlR1NnAEqO1b17d93yyBMmTKBz5840a9YMV1dXmjdvrlvuODIykrZt2+Lq6kqDBg3o2rUrw4YNAyAuLo7hw4dTrVo1vLy8CAgISPdv+b///jtVq1bF09OTBg0aEBYWBsCFCxeoWbMmXl5euLu78/XXXwPwyy+/4OHhgbe3N+7u7rozGkUxNHUGoKTok08gNDRr9u3tDd9+m7a2AQEBLyxpfPHixVTbHj16lCNHjlCgQAHeffddVqxYQZ8+fZg4cSIFCxbk/PnzPHjwAB8fH13BlS+//JICBQpw+PBhAD777DOmTp3KlClT0hTfnTt3CAgIYM+ePbi5ufHTTz/x/vvvc+jQIebPn0+LFi0YOXIkAA8fPgS0S1c//vgjNWvWJDExkcePH6fty1AUPVMJQMnWli1bRvXq1XXbvr6pT3B89913sbfXFk2rXr06ly9fBiA4OJg5c+YAUKhQIVq1aqX7zObNm3n06JGuUEpsbOwLhdzf5NChQ3h5eeHm5gZAjx496N+/P1FRUdSpU4dPP/2UJ0+e4O/vj7+/PwD169dn8ODBtG3blqZNm+Lu7p7m/hRFn1QCUFKU1t/Qs5Pnlw02NTXl6dOnb/yMlJL58+dTv359vcfTtm1batasyfbt25k2bRqLFi1i+fLlzJw5k1OnTrF7927at2/PkCFD6NOnj977V5Q3UfcAlFyvXr16LF26FICIiAh++eUX3XstWrRgxowZumQRFRWVrqUnatSowYkTJzh/XisYHxQURNWqVbGzsyMsLIxixYrRvXt3xo8fr7vMdOHCBTw8PPj444/54IMPOHLkiL7+qIqSLuoMQMn1xo0bR48ePXB1daV48eL4+vpSoEABAEaMGMGECRPw8/PDxMQEIQTjx4+ncuXKKe6rUqVKCCGQUmJra8uFCxdYtmwZXbp0ISEhgSJFirB8+XIA1q5dy4oVK7CwsEAIwaxZs3R9Xrp0CTMzM+zt7fnpp58M80UoyksyvRpoVlKrgRpOSEgIjo6OqR74jEUfa7XEx8eTmJiIlZUVjx49onbt2syYMYOGDRsaNa6soOJKn5we17lz5175P2vo1UAVJVt7+PAhTZs2JTExkZiYGLp06ZKpg7+i5BYqASi5XtGiRTl27Jixw1CUbEfdBFYURcmjVAJQFEXJo1QCUBRFyaMylQCEEIWEEDuEEJeSfxZMpV2iECI0+bE5M30qiqIo+pHZM4ARwC4ppQuwi9QrfT2VUnonP1qk0kZRFEUxoMwmgJZAUPLzIKDVa9oqSrrEx8czefJkKlasiKenJ1WrVmXo0KHEx8cbNI6yZcvi6uqKt7e37nHt2rU3fm7ChAnExcUZIMKUXb16lcKFCxutfyX7y+wwUEcp5b/Jz28Djqm0sxJCHAUSgGlSyk2p7VAIEQgEAjg6OmZ4qdzo6Ohsucxudo6rQIECREVFGTsUnd69e/P06VNCQkKws7MjISGBZcuWce/ePWxtbV9om5iYiKmpaZbEIaUkKChIt+Dbs/7e9F19/vnn9O3b95VYARISEjAz0/8o7Ofjio6ORkqZLf5O0/J9GUNOjysmJiZzxxMp5WsfwE7gdAqPlkDES20fprKPEsk/ywNXAec39SulxMfHR2ZUcHBwhj+blbJzXGfPnn3xxbp1X33Mm6e99/hxyu8vXqy9f/duyu+vXp2meC5evCitra3ltWvXUnx/8eLFskGDBrJVq1aySpUq8vjx4/LSpUuyfv360sPDQ1atWlVu27YtOdTHsl27drJy5crS09NTtm/fXkop5fnz52WNGjWkp6enrFKlivzqq69S7KtMmTLy1KlTL7z26NEjKaWUgJwyZYr09fWV5cqVk+vXr5dSStm/f38JSA8PD+nl5SUfPnwou3XrJnv16iVr164tvby8pJRSbtu2TXp7e0sPDw9Zv359eenSJSml9vfh6ekpAwICpJubm/Tz85NnzpyRUkrZrFkzuXbtWl0sGzZskI0aNXohLimlvHLlinRwcEjxzxQUFCTd3d2lh4eHbNWqlfzvv/+klFLu379fVq1aVXp5eUk3Nze5cuVKKaWUP/zwg3R1dZVeXl7Sw8NDnjt3LsX9pub5uLKTnB7XK/9npZTAUZmG46uU8s1nAFLKVKdMCiH+E0IUl1L+K4QoDtxJZR83k3/+I4QIAaoCl9+YnZQ86/jx47i4uFCwYIrjCgD466+/OHHiBM7OzoC2BHRgYCC9evXi7Nmz1KlTh3PnzrFv3z4ePXqkKy/5bF3+1NbrT0m7du10q42amZkRHBysey9//vwcOXKE/fv306FDB9q2bcu8efOYP38+Bw4ceOEMIDQ0lD179mBjY/PaWgIAJ0+eZPbs2SxdupSgoCC6du3K0aNH+eijj5g+fTrt27cHYN68eQwaNCjN3+3p06cZMWIEx44do3jx4owdO5aPPvqINWvWMH36dIYPH07nzp2RUhIZGQnA8OHDOX/+PMWLFyc2NpbExMQ096dkY2nNFCk9gK+AEcnPRwBfptCmIGCZ/LwwcAlwS8v+1RmA4aR4BmBEa9askV5eXqn+JrR48WL57rvv6rYfPXokLSwsZGJiou61Bg0ayM2bN8vLly/LUqVKyf79+8u1a9fKx48fSymlXL9+vSxfvrwcM2aM3LVrl0xKSkqxrzedAdy9e1dKKWVCQoIE5NOnT3XvRUVF6T7TrVs3OXXqVN325s2bZYMGDXTbiYmJ0sLCQj569EgGBwfLChUqvPJeZGSkTEpKkpUqVZJnz56VZ8+elWXKlJEJCQkvxCVl6mcAs2fPlr169dJt37hxQxYqVEhKKeXMmTOlm5ubnDRpkvzrr790bVq1aiUbNWokZ8+eLS9fvpzi9/Q6Of03bUMz1BlAZm8CTwMaCSEuAQ2TtxFC+AohFia3qQwcFUKcAILR7gGcTXFvipKsatWqXLp06bW/lad0bT0l5cuX58yZMzRq1IidO3fi5eVFTEwMbdu25c8//8TZ2Zlp06YREBCQoVifnRk8uweRkJCQ6ZhfRwjBwIEDmT9/PvPnz6dv3756u//xySefsHnzZooUKcJHH33EmDFjANi4cSOTJ0/m8ePH+Pv7s23bNr30pxhXpu5CSSnvAw1SeP0o0Dv5+QHAIzP9KHmPi4sLLVq04JNPPiEoKAg7OzsSExNZvHgxnTp1eqW9nZ0d3t7eBAUF0aNHD86dO8eJEyeoUaMG4eHhukpgjRs3xsnJiQcPHvDkyRPKly9P9+7dcXFxoUePHnr9M9jZ2REZGZnqQb9GjRr07NmT8+fP4+rq+kItAYDLly/z559/8s4777By5Uo8PDzInz8/AN26dcPNzY3Y2FjOnDmTrrj8/f2ZOnUqt2/fplixYixYsIBGjRoBWsnNihUr4uzsjK2tLUFBQSQkJHDt2jWqVatGtWrVuHz5MsePH6dp06aZ+HaU7EAtBqdkW0FBQYwePRofHx8sLCxISkqiWbNmWFpapth+xYoV9O3bl5kzZ2JmZsayZcsoUqQI27ZtY8QIbYpKYmIiI0eOxMnJiS+++CLF9fpT8vw9AIBZs2ZRt27d18Y/dOhQ6tevT758+VIcqVGkSJFUawkAeHh4sHDhQvr164e1tbWuqA1oyaVJkyY8ffqUIkWKpBrDw4cPKVmypG7b1dWVnTt3Mm3aNBo1aoQQgvLly/PDDz8AMHv2bIKDg7GwsMDS0pI5c+aQmJhI9+7diYiIwMTEhFKlSjFt2rTX/tmVHCKt14qM8VD3AAwnu90DeCanX6PNqODgYPm6f//x8fGycuXK8vDhwwaNK6NUXOmTU+4BKIpiYJs3b8bZ2ZnGjRvj5+dn7HCUHExdAlKUbKhevXqkVg2vRYsWtGihVlRRMk+dASiKouRRKgEoiqLkUSoBKIqi5FEqASiKouRRKgEo2ZZaDjpzpJSMGzeOKlWq4OXlhZubGzNmzADg6NGjvP/++waLZfLkyaxZsybF9yZMmMCwYcNSfK9s2bKcPn06K0PL09QoICXb6tGjB1FRURw7dky3HPSiRYuIjY3F3Nz8hbZZuRw0wPr163F3d9dtp2Wp3s8//5xhw4ZhYWHxyntZtRz089avX8/u3bs5duwYVlZWxMbGcvmytgajr68vK1asyNL+nzdmzBjdDGcl+1BnAEq2dOnSJX7++WfmzJmjO3CYmZkRGBiIra0tS5YsoWHDhrRu3Rp3d3dOnTpFWFgYDRo0wNPTk7feeovff/8dgCdPntC+fXvc3Nzw8vKiQ4cOAFy4cIGaNWvi5eWFu7s7X3/9dbrjFELwxRdf4OfnR/ny5dmwYQMAAwYMAKBWrVp4e3sTERFB9+7d6d27N++88w6+vr4A/P7771StWhVPT08aNGhAWFgYACEhIXh5edG1a1eqVKlCtWrVdKuZNm/enHXr1uli2LhxI40bN34ltvDwcAoXLqybOW1paamraRASEqKLAWDu3Lm4uLjg5+fH+PHjdYVknhWVGTlyJFWrVsXV1ZVjx47Rp08fPD09qV69Ordv3wa0JDxs2DDc3d1xd3dn2LBhulVDP/zwQ+bOnQtAZGQk7dq1w9XVlXr16umSUnocOXKEmjVr4unpSc2aNTly5AgAd+7coWHDhnh4eODh4cHgwYMBOHDgAG+99Rbe3t5UqVKFVatWpbvPXCmtM8aM8VAzgQ0npZnAdevWfePj+TX069atKxcn1wO4e/duqp9Ji7SsBmpjYyPDwsJ0r1WrVk0uXLhQSinlmTNnpIODg7xz5yzFxGQAABafSURBVI7cuHGjbNy4sa7dgwcPpJRSDho0SH7xxRevvP6yMmXKyEqVKkkvLy/p5eUlfXx8XlgNdM6cOVJKKfft2yednJx0nyOF1UB9fHxkdHS0lFLK//77TxYuXFi3zv/ChQtltWrVpJTa3wcgQ0JCpJRSLlmyRDczeNu2bbJevXq6/davX19u2rRJSvniDNJbt27JihUrSmdnZ9m9e3e5bNkyGR8fr9v/s/2dOHFCOjk5yTt37ui+l2eriF65ckUCcsuWLVJKKb/88ktZoEABefz4cSmllP369ZOjR4+WUko5f/582aBBAxkbGytjY2Nl/fr15fz586WUUnbp0kX3PQ0ZMkT26NFDSqn9OylVqpQcOnRoqt/9yyuxxsbGylKlSsmdO3dKKaXcsWOHLFWqlIyNjZUzZsyQgYGBurbP/k5btGihq22QlJQkHz58+Mr3lZ2omcCK8ga1a9fW1QKIiooiNDRUt6Cbm5sb3t7e/PXXX3h5eXHu3DkGDBjAunXrdL8R16lTh4ULFzJ27Fh2796Nvb19qn2tX7+e0NBQQkNDX5mg9Wxxuho1anDr1i1iYmJS3U+7du2wsbEB4NChQ7pr86Bd8goNDdVdXqpQoYJuvaGAgABOnTrF/7V37lFRnecefl5RRGPQNuSAFj0NLVovXLwhtyB4Id6i0cRiqgaD0bQcNHalJuaiidGkpO1KTjSupVYTjKYxNlSbGlBBHQt6VFRQFKL1kniJRsUQBDUqfOePgZ0ZnQFGLjPK96w1C/beH/v77Rlmv/u7/d6SkhIeeeQRzp49S2FhIYWFhRw7dowRI0bcVlf79u05dOgQH374IZ07d+bNN9+0Wc5kMjFs2DDDUyghIcHqeJs2bRg+fDgAvXr1wtfXl+DgYAB69+5ttFoyMzOZNGkS7u7uuLu78/TTT5OZmXlbfVu3bmXy5MkAeHl5MWbMGLvvly0OHz6Mu7s7AweafSgHDRqEu7s7hw8fJjQ0lPT0dGbOnMn69esNI76YmBjmz5/P/Pnz2b17d7WfdVNCjwFo7OJoqjnL8l5eXnVKVWdpB22v79hRO+jNmzeTnp7Oyy+/TH5+Po8//jhhYWFs2rSJ5ORkPvjgAyszttriTDtooFo76ObNm/Pwww/z8MMPk5CQgI+PD5cuXXKoPkvzPTc3NytTPDc3t2qvt7EJCwsjNzeXjIwMVq5cSXJyMtnZ2cyYMYNHH32UzMxMpk2bRmxsLPPnz3e2XKejWwAal8TSDrrqibi8vJxly5ZRWlp6W3lLO2jgNjtoNzc3HnvsMd59910uXLjApUuXOHr0KD4+PkyaNInXXnuN3bt31+s1VNlB2yM0NJT9+/fz5ZdfAti1gwZs2kGvW7eOTz/9lGeeecbm+ffu3ctXX31lbO/bt4+f/OQntz399u/fn/T0dC5evGjouBMGDRrEihUruHHjBjdu3GDFihWGzbQlAwYM4MMPPwSgqKiItWvXOlRPly5duH79upGVbcuWLdy4cYMuXbpw4sQJPD09GTduHO+88w579+6loqKCI0eO8Itf/IJnn32W5557rt4/67uVOrUARGQs8DrmpC8hypwHwFa5IcB7gBuwTCmlvWQ1NaLtoOtmB33x4kUSExMpKSmhZcuWtG7dmnXr1tGsmfVzX1BQEC+88AJhYWF4enoycOBA2rZtW+212WLq1KkcPXqUnj17AvDII48wZcqU28rNnj2bhIQEfvWrX+Hj40NUVFS15x00aJDVjKn8/HxSU1OZPn06ZWVl3HfffXz22We4u7tjMpl45513cHNzo6KigsWLF9OsWTObNtca6pwSsivQBTABfeyUccOc/9cPcAf2o1NCuhzaDtox7jU7aMu/e+2119T48ePv6Dy1Ob8rcbfrcuogsFKqUCl1uIZiIcBRpdRxpdR1YDUwqi71ajRNmYawg541axbBwcF069aNPXv28Kc//alezqtxbRpjEPhnwCmL7dNAv0aoV6O5a2lsO+hFixbV6/k0dwc1BgARyQR8bBx6RSn1z/oWJCJTgakA3t7edzyTpLS0tE6zUBoKV9bVtm3bWq1wbUzKy8tdThNoXY6idTlGbXVdu3atTveTGgOAUmrQHZ/dzBmgo8W2b+U+e/UtBZYC9OnTR0VHR99RpSaTiTv924bElXV5eHi43HL9y5cvu5wm0LocRetyjNrq8vDwMAbd74TGmAaaA/iLyEMi4g6MAz5vhHo1Go1GUw11CgAiMlpETgNhwBcisrFyfwcRSQNQSt0EkoCNQCGwRil1qG6yNRqNRlNX6jQIrJRaC9y2ikMp9Q0wzGI7DUirS10ajUajqV/0SmCNS/Lqq6/yu9/9zthev349IsKhQz82HkeMGMHy5curPU9tfe+rXC9tUVxcXKdpkdV52h85coTRo0fj5+dHnz59iIiIYN26dXdc151gMplo3bq1Vb6Dp556qsa/y8vLY82aNY2g0D7V5RJoKBYsWFAr59hvv/2W2NhYOnfuTFBQELt27bJZLiUlhXbt2hnv/ejRo41jL774In/729/qTfut6ACgcUliYmKsZjds27aNfv36GfvKy8vJzs6ucUC9Pnzv6xoA7HH27FmioqIYM2YMx48fZ8+ePaSmplJSUnJb2Yb22+nWrZthdpeXl2e16tgeNQUAV/IIqi+uXLnCe++9Z9h9V8dLL71EVFQUR44cYdGiRUyYMKFqcextDBo0yHjvLa0x/vCHPzB37lwqKirq7Ros0QFAY5folGhS8lIAuFF+g+iUaFYdMFsVXLlxheiUaD49aM7y9P2174lOieYfhf8A4OKVi0SnRPOvw/8C4FzpOaJTotlwdEOt6g4PD+fEiROcP38eMAeA2bNnGwEgNzcXT09Pww00LS2NiIgIevfuTVhYGDt37gRq73tfxSuvvELPnj3p0qUL2dnZgNnbv7i4mODgYMLDw83Xc+4cTzzxBCEhIQQEBPDWW28Z58jKyjL86JOSkux+6RctWkRMTAwTJ0409vn4+BhP39HR0cyYMYPQ0FBj3v9HH31EQEAAgYGBjB492nh/qvzuIyIirPzuly5dSteuXQkODiYwMNDwHaotKSkpxMbGEhcXR/fu3YmIiODcuXMUFRUxZ84cMjMzCQ4OZvr06YDZqO7111+nb9++zJ0718gR0K9fv9tyBEyaNIkpU6YQHh5O586dmTJlCtevX+ebb76hffv2Vq6qI0eOdOhJuLS0lKefftrITWAZwOfOnWtkeIuMjKS4uNhuzohbSU1NJSoqilatWgHmTGdVbqZXrlwhICCAtDRzb/eaNWv47W9/C5ida1u2bGl3bYc9HnzwQfz8/Ni8ebNDf1dbdADQuCStWrUiJCSErKwsLl++TFlZGUOGDCEvLw+wnk577Ngx5s2bR3p6Onv37mXZsmU2v8AHDhzgj3/8Izt27CAnJ4fi4mKr40VFRYab5Jw5c3jxxRcB8426Xbt25OXlsWPHDsDswDl9+nR2797N3r17SU9PJyMjgx9++IFx48axcOFC8vPziYqK4uTJkzavcd++ffTrV/2ayOPHj5OdnU1aWhoHDx5k1qxZbNq0iQMHDtCjRw+mTZsGwNtvv83MmTPZvn07Bw8eZOjQoQDMnDmTLVu2kJeXR05ODp06dbJZT0FBgVUX0BtvvGEcy8nJ4S9/+QuHDh2iW7duLFy4kAceeIA33njDeHJdsGCB1WeXk5PDvHnzWLp0KXl5eWRlZbFv3z5yc3NZunSpUXbXrl1s2rSJgoICvv76a5YuXUqHDh3o37+/kULyq6++Ys+ePTzxxBPVvleWzJs3j4qKCvLz89mxYwcrVqwgPT2dS5cu8e6775Kbm0teXh7p6em0adOGjRs3UlJSQkFBAfv372fJkiU2z2symaw+s5dffpnS0lIWLlxIUlISQ4cOZdiwYRQVFaGUsnrA6NSpE6dOnbJ1WrZt20ZwcDBRUVF88cUXVsfCwsIaLABoO2iNXUyTTMbvLdxaWG23btHaarutR1urba/WXlbbPm18rLZrQ3R0NNnZ2Xh7exMZGYmbmxv+/v4cOnQIk8nE448/DsDGjRs5duyYlanYzZs3+fbbb62vx4bvvWX3UJs2bQy//NDQUJ5//nmbusrKysjKyjKeesE8b7uwsBBvb29at25tBKdf//rXTJ061aHrtuQ3v/mNYYS2detWhg0bRvv27QFzEAoKCgJ+9LsvKChgxIgRxk1qwIABxMfH8+ijjzJ8+HD8/Pxs1lNlAWGLiIgIOnY0L+UJDQ0lIyOjWs3x8fHG77ZyBKxdu9YY34mLizMssuPj40lNTSUpKYnp06fz+9//nvj4eBYvXkxCQoLN1Jr2yMzM5L333kNE8PT05MknnyQzM5PY2Fh++ctf8tRTTxEbG0t0dDTNmze3yhkRHR1t5D+4ldOnT1vlVGjWrBmrVq0iODiYTp06Ga1GRxgxYgRxcXG0atWK3Nxchg4dyvr1642Wq4+PD//+978dPm9t0C0AjcsSExNDVlYW27ZtM5w3o6Ki2Lx5s1X/v1LKaB1Uvb755hu8vb0dqu9W33t7fdgVFRWICDk5OUZ9x44dswoIloiIzf29evWq0Za4tvkDZsyYweeff46XlxfTpk3j1VdfBczpIufPn09ZWRkxMTGkp6fX6nyWOOr/Xx85D8LDwykvL2f79u2kpKTw7LPP1vmcYNa/c+dOkpKSOH36NFFRURw4cMDIGTF48GAyMzMJCgqymdinVatWt+0/ceIEzZo1o7i4mKtXrwLwwAMPABgW2wAnT540AqklXl5eRpdSz549iYiIsArG165dM47XNzoAaFyWsLAwTp48SWpqqnGzj4qK4v3336ddu3Y89NBDAMTGxrJhwwarGUJVOWItuVPfe09PT65cuWLc+O6//37Cw8NJTv7R1fzUqVOcO3eOLl26cPXqVcPH/7PPPrutq6mKxMRENm/ebNW3ff78ebsDsDExMaSlpRk5eP/6178afvtVfvcJCQmG3/3Nmzc5fvw4ISEhzJo1i9jYWHJzc2t1zbXB09Oz2nwHUHOOgL///e+UlZVx8+ZNVq5cyYABA4xj06ZNY9y4cYSHh9u8cdZU7/Lly1FKcfnyZVavXs3gwYO5fPkyFy5coH///sydO5euXbty8OBBuzkjbiUgIIDDh3/0v/zuu+8YP348q1evJi4uzsr+euzYsSxevBiA7Oxsrl69Su/evW8755kzPxojfP311+zcuZMePXoY+woLC42WXn2jA4DGZfHw8DCawR06dACgb9++nDlzxmr2j7+/P6tWrWLy5MkEBQXRtWtXm324lr73vXv3pnnz5rXyvf/pT3/K+PHjCQgIMAaBly1bRkFBgTHYGxcXR3FxMS1btuSTTz4hMTGRwMBATCaT3X73Dh06sG3bNj799FP8/PwICAhg1KhRdjX16NGD5ORkBg8eTGBgIPv37zdyGCxYsIDu3bsTGRnJwoULefPNNykvL2fSpEkEBAQQFBTE2bNn7T5J3zoGMGzYMJvlLBk4cCBlZWUEBQXZbf1MnTqVwMBAIiMj6dmzJ4GBgVY3yb59+xIbG0vXrl3p2LGjVXfZuHHj+O6770hMTKxWx5IlS/D19TVeS5YsYfbs2SilCAgIICwsjIkTJzJkyBC+//57HnvsMQIDA+nRowfe3t6MGTOG/Px8wsLCCAoKIiQkxMgZcStjxoxh48aNxnZCQgIJCQlERkYyZ84czp07Z9z0k5OTMZlM+Pv7k5iYyMqVK41cDM888wyff242RFi0aBHdu3cnKCiIkSNH8tZbbxk3fKUUW7ZsYdSoBjJQrq1vtDNeOh9A49FU8gHUl+/93e4j39jY0hUfH28kirdFVlaW6t69u6qoqGhUXTURGxur9u3b1wBqfqRK14YNG9SECRPslqtrPgA9CKxpUsyaNYvt27dz/fp1/Pz8rGakaFyHyZMnk5GRwUcffWR3DMVZvP/++/znP/+pkwlbbSkpKeHtt99usPPrAKBpUmjfe9chJSXF7rGaVng7E39/f/z9/RulrrFjxzbo+fUYgMYKZWfRkkajcS3q47uqA4DGwMPDw1jAotFoXBelFEVFRVZTdO8E3QWkMfD19eX06dNcuHDB2VIMrl27Vud/8oZA63IMrcsxaqPLw8MDX1/fOtWjA4DGoEWLFsbcelfBZDI1ymCbo2hdjqF1OUZj6dJdQBqNRtNEqWtGsLEickhEKkSkTzXlvhKRfBHJExHH7PA0Go1G0yDUtQvoIDAGsG2dZ02MUupizcU0Go1G0xjUNSVkIdg3u9JoNBqN69JYg8AK2CQiCliilLK7/FJEpgJVhiClInLYXtka8AJcscWhdTmG1uUYWpdj3Iu6/ru2BWsMACKSCfjYOPSKUuqftawnUil1RkT+C8gQkS+VUjYNriuDQ53X54vIHqWU3XEJZ6F1OYbW5Rhal2M0dV01BgCl1KC6VqKUOlP587yIrAVCgIbJcKDRaDSaWtHg00BF5D4Rub/qdyAW8+CxRqPRaJxIXaeBjhaR00AY8IWIbKzc30FE0iqLeQPZIrIf2A18oZSqXWbwuuGqNo9al2NoXY6hdTlGk9Yl2vdFo9FomiZ6JbBGo9E0UXQA0Gg0mibKPR0ARGSeiByotKDYJCK3J/l0AiLyZxH5slLbWhFp52xNUHtrj0bSMkREDovIURGZ5UwtlojIByJyXkRcaiKDiHQUka0iUlD5GT7nbE0AIuIhIrtFZH+lrrnO1lSFiLiJSK6IrHe2Fksa0zrnng4AwJ+VUoFKqWBgPTDH2YIqyQB6KKUCgSPAS07WU0WVtYdTp+iKiBuwCBgKdAOeFJFuztRkQQowxNkibHATeF4p1Q0IBf7HRd6zH4ABSqkgIBgYIiKhTtZUxXNAobNF2CFGKRXc0GsB7ukAoJQqsdi8D/OKZKejlNqklLpZubkTqJupdz2hlCpUSt3pyuv6JAQ4qpQ6rpS6DqwGRjlZEwCVCxgvOVvHrSilziql9lX+fhnzje1nzlUFlXnKSys3W1S+nP49FBFfYDiwzNlanMk9HQAARORNETkFjMd1WgCWJADpzhbhYvwMOGWxfRoXuJndLYjIz4GewC7nKjFT2dWSB5wHMpRSrqDrf4EXgApnC7FBlXXO3kprnAbjrg8AIpIpIgdtvEYBKKVeUUp1BD4GklxFV2WZVzA33T92JV2auxcRaQOkAjNuaQE7DaVUeWU3rC8QIiI9nKlHREYA55VSe52poxoilVK9MHeB/o+IRDVURXd9RjAHrCo+BtKA1xpQjkFNukRkEjACGKgacTFGfVh7NAJngI4W276V+zTVICItMN/8P1ZK/cPZem5FKVUsIlsxj6E4cxA9AhgpIsMAD8BTRFYppSY4UZNBY1rn3PUtgOoQEX+LzVHAl87SYomIDMHc/ByplLribD0uSA7gLyIPiYg7MA743MmaXBoxe7IvBwqVUu84W08VIvJg1Sw3EWkFDMbJ30Ol1EtKKV+l1M8x/29tcZWbf2Nb59zTAQBIruzeOID5jXSJqXHA+8D9mJ1R80RksbMFgX1rj8amcoA8CdiIeTBzjVLqkDO03IqIfAL8H9BFRE6LyGRna6okApgIDKj8n8qrfMJ1Nu2BrZXfwRzMYwAuNe3SxWhU6xxtBaHRaDRNlHu9BaDRaDQaO+gAoNFoNE0UHQA0Go2miaIDgEaj0TRRdADQaDSaJooOABqNRtNE0QFAo9Fomij/D5ZMqSmPBhxGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the output\n",
    "x_array = sess.run(x_vals)\n",
    "plt.plot(x_array, hinge_y_out, 'b-', label='Hinge Loss')\n",
    "plt.plot(x_array, xentropy_y_out, 'r--', label='Cross Entropy Loss')\n",
    "plt.plot(x_array, xentropy_sigmoid_y_out, 'k-.', label='Cross Entropy Sigmoid Loss')\n",
    "plt.plot(x_array, xentropy_weighted_y_out, 'g:', label='Weighted Cross Entropy Loss (x0.5)')\n",
    "plt.ylim(-1.5, 3)\n",
    "plt.legend(loc='lower right', prop={'size': 11})\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Softmax entropy and Sparse Entropy ----------> Classification\n",
    "\n",
    "Since it is hard to graph mutliclass loss functions, we will show how to get the output instead"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.1601256]\n",
      "[0.00012564]\n"
     ]
    }
   ],
   "source": [
    "# Softmax entropy loss\n",
    "# L = -actual * (log(softmax(pred))) - (1-actual)(log(1-softmax(pred)))\n",
    "unscaled_logits = tf.constant([[1., -3., 10.]])\n",
    "target_dist = tf.constant([[0.1, 0.02, 0.88]])\n",
    "softmax_xentropy = tf.nn.softmax_cross_entropy_with_logits_v2(logits=unscaled_logits,\n",
    "                                                              labels=target_dist)\n",
    "print(sess.run(softmax_xentropy))\n",
    "\n",
    "# Sparse entropy loss\n",
    "# Use when classes and targets have to be mutually exclusive\n",
    "# L = sum( -actual * log(pred) )\n",
    "unscaled_logits = tf.constant([[1., -3., 10.]])\n",
    "sparse_target_dist = tf.constant([2])\n",
    "sparse_xentropy =  tf.nn.sparse_softmax_cross_entropy_with_logits(logits=unscaled_logits,\n",
    "                                                                  labels=sparse_target_dist)\n",
    "print(sess.run(sparse_xentropy))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
